diff --git "a/evals/core_9mcqa/task-010-openbookqa:mc-requests.jsonl" "b/evals/core_9mcqa/task-010-openbookqa:mc-requests.jsonl"
new file mode 100644--- /dev/null
+++ "b/evals/core_9mcqa/task-010-openbookqa:mc-requests.jsonl"
@@ -0,0 +1,2000 @@
+{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 0, "native_id": "8-343", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 0, "native_id": "8-343", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 0, "native_id": "8-343", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\n A. make more phone calls\n B. quit eating lunch out\n C. buy less with monopoly money\n D. have lunch with friends\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 0, "native_id": "8-343", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 1, "native_id": "1129", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 1, "native_id": "1129", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 1, "native_id": "1129", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is most likely going to be fog around:\n A. a marsh\n B. a tundra\n C. the plains\n D. a desert\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 1, "native_id": "1129", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 2, "native_id": "880", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 2, "native_id": "880", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 2, "native_id": "880", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Predators eat\n A. lions\n B. humans\n C. bunnies\n D. grass\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 2, "native_id": "880", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 3, "native_id": "7-999", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 3, "native_id": "7-999", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 3, "native_id": "7-999", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\n A. roots may be split\n B. roots may begin to die\n C. parts may break the concrete\n D. roots may fall apart\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 3, "native_id": "7-999", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 4, "native_id": "8-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 4, "native_id": "8-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 4, "native_id": "8-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An electric car runs on electricity via\n A. gasoline\n B. a power station\n C. electrical conductors\n D. fuel\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 4, "native_id": "8-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 5, "native_id": "9-794", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 5, "native_id": "9-794", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 5, "native_id": "9-794", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\n A. oxygen\n B. nitrogen\n C. carbon\n D. rain\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 5, "native_id": "9-794", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 6, "native_id": "9-1163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 6, "native_id": "9-1163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 6, "native_id": "9-1163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: an electric car contains a motor that runs on\n A. gas\n B. hydrogen\n C. ions\n D. plutonium\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 6, "native_id": "9-1163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 7, "native_id": "9-322", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 7, "native_id": "9-322", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 7, "native_id": "9-322", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\n A. moons gravity\n B. human planet rotation\n C. global warming\n D. moon rotation\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 7, "native_id": "9-322", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 8, "native_id": "7-1140", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 8, "native_id": "7-1140", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 8, "native_id": "7-1140", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The summer solstice in the northern hemisphere is four months before\n A. May\n B. July\n C. April\n D. October\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 8, "native_id": "7-1140", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 9, "native_id": "7-903", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 9, "native_id": "7-903", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 9, "native_id": "7-903", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The main component in dirt is\n A. microorganisms\n B. broken stones\n C. pollution\n D. bacteria\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 9, "native_id": "7-903", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 10, "native_id": "7-511", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 10, "native_id": "7-511", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 10, "native_id": "7-511", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It's easier for human's to survive in:\n A. a cave\n B. the ocean.\n C. a town\n D. alone\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 10, "native_id": "7-511", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 11, "native_id": "9-937", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 11, "native_id": "9-937", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 11, "native_id": "9-937", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cactus stem is used to store\n A. fruit\n B. liquid\n C. food\n D. spines\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 11, "native_id": "9-937", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 12, "native_id": "8-201", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 12, "native_id": "8-201", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 12, "native_id": "8-201", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\n A. an eagle\n B. a cow\n C. a gecko\n D. a deer\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 12, "native_id": "8-201", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 13, "native_id": "1618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 13, "native_id": "1618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 13, "native_id": "1618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The chance of wildfires is increased by\n A. parched foliage\n B. torrential rain\n C. lush foliage\n D. careful fire maintenance\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 13, "native_id": "1618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 14, "native_id": "758", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 14, "native_id": "758", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 14, "native_id": "758", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A positive effect of burning biofuel is\n A. shortage of crops for the food supply\n B. an increase in air pollution\n C. powering the lights in a home\n D. deforestation in the amazon to make room for crops\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 14, "native_id": "758", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 15, "native_id": "7-414", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 15, "native_id": "7-414", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 15, "native_id": "7-414", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\n A. wind power will be expensive\n B. gas costs will rise\n C. oil costs will be maintained\n D. gasoline will be needed less\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 15, "native_id": "7-414", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 16, "native_id": "9-675", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 16, "native_id": "9-675", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 16, "native_id": "9-675", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\n A. sun grafts\n B. sunlight shields\n C. panels collecting sunlight\n D. solar bees\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 16, "native_id": "9-675", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 17, "native_id": "9-163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 17, "native_id": "9-163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 17, "native_id": "9-163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Mola Mola might live where?\n A. Lake Michigan\n B. The Mississippi River\n C. Bay of Bengal\n D. Lake Eerie\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 17, "native_id": "9-163", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 18, "native_id": "1032", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 18, "native_id": "1032", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 18, "native_id": "1032", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which requires energy to move?\n A. weasel\n B. willow\n C. mango\n D. poison ivy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 18, "native_id": "1032", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 19, "native_id": "889", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 19, "native_id": "889", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 19, "native_id": "889", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal that only eats plants is a\n A. rat\n B. moth\n C. chimpanzee\n D. pig\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 19, "native_id": "889", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 20, "native_id": "1160", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 20, "native_id": "1160", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 20, "native_id": "1160", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\n A. Hanoi\n B. Athens\n C. Baghdad\n D. Phoenix\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 20, "native_id": "1160", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 21, "native_id": "9-298", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 21, "native_id": "9-298", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 21, "native_id": "9-298", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of conservation is avoiding the use of\n A. gasoline\n B. air\n C. snow\n D. clothes\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 21, "native_id": "9-298", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 22, "native_id": "1189", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 22, "native_id": "1189", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 22, "native_id": "1189", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can feathers on Spheniscidae be used for?\n A. keeping warm\n B. flying\n C. sleeping\n D. eating\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 22, "native_id": "1189", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 23, "native_id": "8-395", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 23, "native_id": "8-395", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 23, "native_id": "8-395", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation can cause\n A. More fresh water for people to drink\n B. Lower Life Expectancy in Countries\n C. More food for more people\n D. More space for places to people to live\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 23, "native_id": "8-395", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 24, "native_id": "7-238", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 24, "native_id": "7-238", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 24, "native_id": "7-238", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Shining a light through a diamond can\n A. make a lot of bright lights shine\n B. summon a brilliant wave of color\n C. heat up a room\n D. make a lot of money\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 24, "native_id": "7-238", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 25, "native_id": "7-372", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 25, "native_id": "7-372", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 25, "native_id": "7-372", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\n A. its snout\n B. its gills\n C. its nose\n D. its belly\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 25, "native_id": "7-372", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 26, "native_id": "8-35", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 26, "native_id": "8-35", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 26, "native_id": "8-35", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would stop a car quicker?\n A. a wheel with wet brake pads\n B. a wheel without brake pads\n C. a wheel with worn brake pads\n D. a wheel with dry brake pads\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 26, "native_id": "8-35", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 27, "native_id": "9-271", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 27, "native_id": "9-271", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 27, "native_id": "9-271", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\n A. the circulatory system\n B. the digestive system\n C. the school system\n D. central nervous system\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 27, "native_id": "9-271", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 28, "native_id": "9-409", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 28, "native_id": "9-409", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 28, "native_id": "9-409", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\n A. gone\n B. breaking\n C. falling\n D. moving upwards\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 28, "native_id": "9-409", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 29, "native_id": "530", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 29, "native_id": "530", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 29, "native_id": "530", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When it's flying, a plane has no friction with the\n A. wings\n B. ground\n C. air\n D. clouds\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 29, "native_id": "530", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 30, "native_id": "1426", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 30, "native_id": "1426", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 30, "native_id": "1426", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To grow plants require\n A. acid rain\n B. pesticides\n C. shafts of sunlight\n D. moonbeam rays\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 30, "native_id": "1426", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 31, "native_id": "8-466", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 31, "native_id": "8-466", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 31, "native_id": "8-466", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the best way to guess a babies eye color?\n A. The surroundings they are born in.\n B. Their parents usual diet.\n C. Just take a random guess.\n D. The genealogy records of their family.\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 31, "native_id": "8-466", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 32, "native_id": "1577", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 32, "native_id": "1577", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 32, "native_id": "1577", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal eats plants?\n A. eagles\n B. robins\n C. owls\n D. leopards\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 32, "native_id": "1577", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 33, "native_id": "8-257", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 33, "native_id": "8-257", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 33, "native_id": "8-257", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a hypothesis?\n A. The ice caps will completely melt if global warming continues\n B. The earth is round\n C. The earth revolves around the sun\n D. Gravity causes objects to fall\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 33, "native_id": "8-257", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 34, "native_id": "378", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 34, "native_id": "378", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 34, "native_id": "378", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What explains the characteristic lunar formations?\n A. remains of ancient ponds\n B. many collisions that have occured\n C. volcanic explosions over millions of years\n D. sink holes due to the moons porous nature\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 34, "native_id": "378", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 35, "native_id": "8-41", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 35, "native_id": "8-41", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 35, "native_id": "8-41", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tadpoles start their lives as\n A. Water animals\n B. Frogs\n C. Ants\n D. College Students\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 35, "native_id": "8-41", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 36, "native_id": "9-540", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 36, "native_id": "9-540", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 36, "native_id": "9-540", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\n A. the apple sitting on a sunny sidewalk\n B. the apple in the freezer\n C. the apple sitting on the shaded stoop\n D. the apple in a closet\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 36, "native_id": "9-540", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 37, "native_id": "266", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 37, "native_id": "266", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 37, "native_id": "266", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is used for sensing visual things?\n A. nerves\n B. tibia\n C. nostril\n D. cornea\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 37, "native_id": "266", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 38, "native_id": "1309", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 38, "native_id": "1309", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 38, "native_id": "1309", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They studied the soil by using\n A. plants\n B. a telescope\n C. roots\n D. a microscope\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 38, "native_id": "1309", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 39, "native_id": "7-1197", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 39, "native_id": "7-1197", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 39, "native_id": "7-1197", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill's arm got cold when he put it inside the\n A. refrigerator\n B. room\n C. jacket\n D. oven\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 39, "native_id": "7-1197", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 40, "native_id": "7-891", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 40, "native_id": "7-891", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 40, "native_id": "7-891", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A recyclable material can be\n A. transformed\n B. traded\n C. thrown away\n D. used more times\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 40, "native_id": "7-891", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 41, "native_id": "1180", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 41, "native_id": "1180", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 41, "native_id": "1180", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is different about birth in humans and chickens?\n A. Mother\n B. Fertilization\n C. Father\n D. the hard shell\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 41, "native_id": "1180", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 42, "native_id": "1204", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 42, "native_id": "1204", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 42, "native_id": "1204", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these situations is an example of pollutants?\n A. plastic bags floating in the ocean\n B. mallard ducks floating on a lake\n C. cottonwood seeds floating in the air\n D. cirrus clouds floating in the sky\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 42, "native_id": "1204", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 43, "native_id": "7-52", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 43, "native_id": "7-52", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 43, "native_id": "7-52", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Human reproduction requires\n A. eggs with shells\n B. nest incubation\n C. a nest\n D. a womb\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 43, "native_id": "7-52", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 44, "native_id": "1759", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 44, "native_id": "1759", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 44, "native_id": "1759", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Thermometers\n A. can help you monitor a fever\n B. indicate levels of mercury in the blood\n C. read exactly at 98.6 degrees\n D. are used only for babies\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 44, "native_id": "1759", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 45, "native_id": "9-655", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 45, "native_id": "9-655", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 45, "native_id": "9-655", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\n A. someone would turn up the room heater\n B. someone would turn up the air conditioner\n C. someone would turn up the music\n D. someone would turn on the light\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 45, "native_id": "9-655", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 46, "native_id": "132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 46, "native_id": "132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 46, "native_id": "132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would happen when balloons heat up?\n A. they get bigger\n B. they get smaller\n C. nothing happens\n D. they fall down\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 46, "native_id": "132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 47, "native_id": "8-79", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 47, "native_id": "8-79", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 47, "native_id": "8-79", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\n A. expand\n B. melt\n C. shrink\n D. fall\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 47, "native_id": "8-79", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 48, "native_id": "1835", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 48, "native_id": "1835", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 48, "native_id": "1835", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seals are most likely to be found in what type of environment?\n A. desert\n B. arctic\n C. Mediterranean\n D. tropical\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 48, "native_id": "1835", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 49, "native_id": "9-149", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 49, "native_id": "9-149", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 49, "native_id": "9-149", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When the eggs hatch, the offspring are\n A. killed\n B. hurt\n C. born\n D. cold\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 49, "native_id": "9-149", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 50, "native_id": "695", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 50, "native_id": "695", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 50, "native_id": "695", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some berries may be eaten by\n A. a bear or person\n B. a bear or shark\n C. a bear or lion\n D. a bear or wolf\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 50, "native_id": "695", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 51, "native_id": "8-179", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 51, "native_id": "8-179", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 51, "native_id": "8-179", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person has a chance to experience an equinox\n A. weekly\n B. monthly\n C. annually\n D. biannually\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 51, "native_id": "8-179", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 52, "native_id": "7-50", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 52, "native_id": "7-50", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 52, "native_id": "7-50", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Overpopulation of an organism can\n A. strain the resources of an ecosystem\n B. cause boundless growth of resources\n C. lead to extinction of the organism\n D. cause the ecosystem to flourish\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 52, "native_id": "7-50", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 53, "native_id": "508", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 53, "native_id": "508", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 53, "native_id": "508", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To improve health, what is a good strategy?\n A. high risk lifestyle\n B. restaurant food\n C. business trip\n D. a spa trip\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 53, "native_id": "508", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 54, "native_id": "1674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 54, "native_id": "1674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 54, "native_id": "1674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A girl and her mom have the same\n A. date of birth\n B. shirt\n C. number of toenails\n D. hair length\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 54, "native_id": "1674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 55, "native_id": "163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 55, "native_id": "163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 55, "native_id": "163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The transportation with the most mass is likely a\n A. commercial plane\n B. private plane\n C. bus\n D. private car\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 55, "native_id": "163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 56, "native_id": "7-49", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 56, "native_id": "7-49", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 56, "native_id": "7-49", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit may enjoy\n A. meat\n B. compost\n C. peas\n D. pebbles\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 56, "native_id": "7-49", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 57, "native_id": "8-393", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 57, "native_id": "8-393", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 57, "native_id": "8-393", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\n A. the deer are moved to a zoo\n B. the trail is expanded\n C. the mall has a nature park in it\n D. the wildlife environment is destroyed\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 57, "native_id": "8-393", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 58, "native_id": "788", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 58, "native_id": "788", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 58, "native_id": "788", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Owls are likely to hunt at\n A. 3pm\n B. 2am\n C. 6pm\n D. 7am\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 58, "native_id": "788", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 59, "native_id": "9-29", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 59, "native_id": "9-29", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 59, "native_id": "9-29", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could be a positive aspect of a tree being cut down?\n A. the plants that were under the tree will have access to more light\n B. the squirrels that were in that tree will have an easier time getting to their home\n C. Plants under the tree will get cooled off by the shade\n D. The sun will shine brighter than before\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 59, "native_id": "9-29", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 60, "native_id": "9-368", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 60, "native_id": "9-368", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 60, "native_id": "9-368", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Birds carrying away fruit helps the tree\n A. grow\n B. fertilize\n C. reproduce\n D. conquer\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 60, "native_id": "9-368", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 61, "native_id": "7-671", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 61, "native_id": "7-671", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 61, "native_id": "7-671", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\n A. the UFO is calling\n B. the UFO had been close\n C. the UFO is approaching\n D. the UFO is leaving\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 61, "native_id": "7-671", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 62, "native_id": "1272", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 62, "native_id": "1272", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 62, "native_id": "1272", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited behavior is exhibited when\n A. bears take a long winter sleep\n B. dogs sit on command\n C. seals clap for their trainers\n D. rats navigate thru a maze\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 62, "native_id": "1272", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 63, "native_id": "648", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 63, "native_id": "648", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 63, "native_id": "648", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What likely explains deforestation?\n A. Increased insect populations\n B. Clearing for farming\n C. reduction in rainfall\n D. More carbon dioxide\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 63, "native_id": "648", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 64, "native_id": "9-1180", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 64, "native_id": "9-1180", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 64, "native_id": "9-1180", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\n A. steak\n B. blood\n C. nice weather\n D. taking food\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 64, "native_id": "9-1180", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 65, "native_id": "9-227", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 65, "native_id": "9-227", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 65, "native_id": "9-227", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The surface of the moon contains\n A. dogs\n B. water\n C. high peaks\n D. humans\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 65, "native_id": "9-227", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 66, "native_id": "1582", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 66, "native_id": "1582", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 66, "native_id": "1582", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\n A. four boxes\n B. eight boxes\n C. two boxes\n D. six boxes\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 66, "native_id": "1582", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 67, "native_id": "8-125", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 67, "native_id": "8-125", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 67, "native_id": "8-125", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\n A. asks for milk\n B. asks for television\n C. asks for water\n D. asks for sleep\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 67, "native_id": "8-125", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 68, "native_id": "1923", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 68, "native_id": "1923", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 68, "native_id": "1923", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\n A. roots\n B. tensile strength\n C. plant temperature\n D. plant color\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 68, "native_id": "1923", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 69, "native_id": "9-229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 69, "native_id": "9-229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 69, "native_id": "9-229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\n A. rainy sky to fly in\n B. fast truck to drive\n C. a car with gasoline\n D. surfaces to reflect sound off\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 69, "native_id": "9-229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 70, "native_id": "1702", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 70, "native_id": "1702", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 70, "native_id": "1702", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\n A. puppies have soft fur\n B. kittens look like their parents\n C. all mammals are born with fur\n D. baby rats are mostly bald\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 70, "native_id": "1702", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 71, "native_id": "8-260", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 71, "native_id": "8-260", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 71, "native_id": "8-260", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Decaying vegetation is part of the process that\n A. enables nuclear power to function\n B. enables to emitting of light beams\n C. enables gas powered motors to operate\n D. enables windmills to power electric grids\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 71, "native_id": "8-260", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 72, "native_id": "9-491", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 72, "native_id": "9-491", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 72, "native_id": "9-491", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\n A. melted\n B. dehydrated\n C. bloated\n D. salted\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 72, "native_id": "9-491", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 73, "native_id": "75", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 73, "native_id": "75", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 73, "native_id": "75", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\n A. inverse\n B. reverse\n C. direct\n D. equal\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 73, "native_id": "75", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 74, "native_id": "1215", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 74, "native_id": "1215", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 74, "native_id": "1215", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A fallen leaf\n A. will turn into a tree\n B. will become bright green\n C. will begin to recycle the nutrients that made up its structure\n D. is likely to continue to grow\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 74, "native_id": "1215", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 75, "native_id": "8-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 75, "native_id": "8-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 75, "native_id": "8-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Over a period of time the weather can change\n A. The color of my hair\n B. The way I walk\n C. The size of a statue\n D. The sound a computer makes\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 75, "native_id": "8-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 76, "native_id": "7-988", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 76, "native_id": "7-988", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 76, "native_id": "7-988", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plant growth may cause\n A. an uptick in the number of leaves\n B. a surge in leaf disease\n C. a gradual decrease in leaves\n D. a rapid decline of the leaves\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 76, "native_id": "7-988", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 77, "native_id": "9-1139", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 77, "native_id": "9-1139", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 77, "native_id": "9-1139", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\n A. ice cube\n B. water\n C. electrical transformer\n D. grass\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 77, "native_id": "9-1139", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 78, "native_id": "1545", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 78, "native_id": "1545", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 78, "native_id": "1545", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which substance is capable of dripping?\n A. Oxygen\n B. Juice\n C. Wood\n D. Lightning\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 78, "native_id": "1545", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 79, "native_id": "7-664", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 79, "native_id": "7-664", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 79, "native_id": "7-664", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If bacon is left too long on a hot stove top\n A. it will be cooked perfectly\n B. it will be bacteria laden\n C. it will become blackened\n D. it will be left raw\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 79, "native_id": "7-664", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 80, "native_id": "8-53", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 80, "native_id": "8-53", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 80, "native_id": "8-53", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\n A. set to calories\n B. set to volume\n C. set to kilometers\n D. set to width\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 80, "native_id": "8-53", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 81, "native_id": "7-1044", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 81, "native_id": "7-1044", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 81, "native_id": "7-1044", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: are explosions safe?\n A. they could harm living things\n B. they are very safe\n C. they cause nothing serious\n D. none of these\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 81, "native_id": "7-1044", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 82, "native_id": "7-1122", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 82, "native_id": "7-1122", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 82, "native_id": "7-1122", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lowest temperature on the trip was at\n A. the mountain pass\n B. the plain\n C. the large hill\n D. the canyon\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 82, "native_id": "7-1122", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 83, "native_id": "9-79", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 83, "native_id": "9-79", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 83, "native_id": "9-79", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\n A. they are easier to make\n B. they let in less light\n C. they are cheaper to produce\n D. they are much stronger\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 83, "native_id": "9-79", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 84, "native_id": "7-157", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 84, "native_id": "7-157", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 84, "native_id": "7-157", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Ocean water contains\n A. copious amounts of seltzer\n B. scant amounts of sodium chloride\n C. scant amounts of carbonation\n D. copious amounts of the combination of Na and Cl\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 84, "native_id": "7-157", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 85, "native_id": "9-1164", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 85, "native_id": "9-1164", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 85, "native_id": "9-1164", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\n A. blood\n B. water\n C. prey\n D. spots\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 85, "native_id": "9-1164", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 86, "native_id": "8-63", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 86, "native_id": "8-63", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 86, "native_id": "8-63", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Beak shape can influence a bird's ability\n A. to give birth to live young\n B. to mate with it's partner\n C. to fly to warmer climates\n D. to chew up certain worms\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 86, "native_id": "8-63", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 87, "native_id": "8-308", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 87, "native_id": "8-308", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 87, "native_id": "8-308", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some blind people have demonstrated bat-like skills by:\n A. sensing shapes by light and shadows\n B. having a unusually strong sense of smell\n C. sensing nearby objects by temperature change\n D. using sound to 'see'\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 87, "native_id": "8-308", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 88, "native_id": "326", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 88, "native_id": "326", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 88, "native_id": "326", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\n A. A session\n B. A Match\n C. A Season\n D. Autumn\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 88, "native_id": "326", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 89, "native_id": "1184", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 89, "native_id": "1184", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 89, "native_id": "1184", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds provide new plants with\n A. life sustaining elements\n B. essentials for photosynthesis\n C. water and hydrogen\n D. storage for roots\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 89, "native_id": "1184", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 90, "native_id": "359", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 90, "native_id": "359", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 90, "native_id": "359", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\n A. ecru\n B. red\n C. black\n D. navy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 90, "native_id": "359", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 91, "native_id": "9-350", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 91, "native_id": "9-350", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 91, "native_id": "9-350", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The salamander could eat a large amounts of what?\n A. fettuccine if left around\n B. waxy leaves from certain plants\n C. dead carcass meat from livestock\n D. six legged winged organisms\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 91, "native_id": "9-350", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 92, "native_id": "7-140", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 92, "native_id": "7-140", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 92, "native_id": "7-140", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person can see\n A. a radio recording\n B. an emotion\n C. a written message\n D. an abstract idea\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 92, "native_id": "7-140", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 93, "native_id": "591", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 93, "native_id": "591", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 93, "native_id": "591", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When looking for good soil for plants, typically what is optimal?\n A. malleable and nutritious\n B. dry and sandy\n C. grainy and bitter\n D. compact and hard\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 93, "native_id": "591", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 94, "native_id": "7-391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 94, "native_id": "7-391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 94, "native_id": "7-391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals are drawn to\n A. gold\n B. houses\n C. feeders\n D. Carbon Dioxide\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 94, "native_id": "7-391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 95, "native_id": "1672", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 95, "native_id": "1672", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 95, "native_id": "1672", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Squirrels spend their fall\n A. looking for pretty leaves to collect\n B. stockpiling rocks for fighting in the winter\n C. stockpiling pecans for the frigid months\n D. collecting twigs to keep warm\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 95, "native_id": "1672", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 96, "native_id": "9-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 96, "native_id": "9-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 96, "native_id": "9-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rainbows are always found after what?\n A. A fire\n B. A tornado\n C. Rainfall\n D. Cereal\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 96, "native_id": "9-464", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 97, "native_id": "9-983", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 97, "native_id": "9-983", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 97, "native_id": "9-983", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Crop rotation has a positive impact on what?\n A. government mentality\n B. dirt quality\n C. town economies\n D. crop watering\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 97, "native_id": "9-983", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 98, "native_id": "9-179", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 98, "native_id": "9-179", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 98, "native_id": "9-179", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the best method for detecting texture is\n A. rubbing it\n B. seeing it\n C. hearing it\n D. tasting it\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 98, "native_id": "9-179", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 99, "native_id": "7-942", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 99, "native_id": "7-942", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 99, "native_id": "7-942", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cold-blooded animals are often\n A. fast\n B. large\n C. hairless\n D. slow\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 99, "native_id": "7-942", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 100, "native_id": "7-100", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 100, "native_id": "7-100", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 100, "native_id": "7-100", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grey clouds can bring\n A. sunlight\n B. falling water molecules\n C. blooming flowers\n D. drought conditions\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 100, "native_id": "7-100", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 101, "native_id": "9-30", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 101, "native_id": "9-30", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 101, "native_id": "9-30", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is considered a predator?\n A. ant\n B. snake\n C. elephant\n D. giraffe\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 101, "native_id": "9-30", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 102, "native_id": "1709", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 102, "native_id": "1709", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 102, "native_id": "1709", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pollinators\n A. enable plants to continue flourishing\n B. play an unimportant role in the reproduction process\n C. are useless to plants\n D. are considered unwanted pests\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 102, "native_id": "1709", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 103, "native_id": "8-491", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 103, "native_id": "8-491", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 103, "native_id": "8-491", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\n A. have their own dirt\n B. have their own sunlight\n C. have a lot of sand\n D. contain their necessary nutrition\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 103, "native_id": "8-491", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 104, "native_id": "44", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 104, "native_id": "44", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 104, "native_id": "44", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of characteristics are people not born with?\n A. genetics\n B. skills\n C. physical attributes\n D. height\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 104, "native_id": "44", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 105, "native_id": "1023", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 105, "native_id": "1023", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 105, "native_id": "1023", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A spinning object is used to make\n A. steam\n B. heat\n C. water\n D. electricity\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 105, "native_id": "1023", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 106, "native_id": "1911", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 106, "native_id": "1911", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 106, "native_id": "1911", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: One of the negative consequences of offshore oil platforms is\n A. evaporation of the surrounding water\n B. discharge of liquid petroleum in the surrounding sea\n C. improvement in the conditions of sea life\n D. increase in the birthrate of sea birds\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 106, "native_id": "1911", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 107, "native_id": "429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 107, "native_id": "429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 107, "native_id": "429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\n A. 1000\n B. 250\n C. 5000\n D. 300\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 107, "native_id": "429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 108, "native_id": "8-49", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 108, "native_id": "8-49", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 108, "native_id": "8-49", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\n A. getting a new child\n B. yelling at the mud\n C. asking the child to stop\n D. turning on the hose\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 108, "native_id": "8-49", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 109, "native_id": "520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 109, "native_id": "520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 109, "native_id": "520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthquakes\n A. only happen in California\n B. cause solar and lunar eclipses\n C. will break your vases\n D. make bridges much safer\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 109, "native_id": "520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 110, "native_id": "7-1128", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 110, "native_id": "7-1128", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 110, "native_id": "7-1128", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A seismograph can accurately describe\n A. how rough the footing will be\n B. how bad the weather will be\n C. how stable the ground will be\n D. how shaky the horse will be\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 110, "native_id": "7-1128", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 111, "native_id": "7-394", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 111, "native_id": "7-394", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 111, "native_id": "7-394", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\n A. the sun is always bright\n B. the moon is brighter than stars\n C. the moon is brighter than a floodlight\n D. the sun is darker than the moon\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 111, "native_id": "7-394", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 112, "native_id": "9-1166", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 112, "native_id": "9-1166", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 112, "native_id": "9-1166", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant needs a specific climate to grow and\n A. wither\n B. persist\n C. die\n D. decay\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 112, "native_id": "9-1166", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 113, "native_id": "7-884", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 113, "native_id": "7-884", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 113, "native_id": "7-884", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Banging on a drum causes\n A. music to be loud\n B. music to be appealing\n C. reverberations to strike the eardrum\n D. concerts to sell out\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 113, "native_id": "7-884", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 114, "native_id": "9-501", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 114, "native_id": "9-501", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 114, "native_id": "9-501", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What may have been formed by a volcano?\n A. Mt. McKinley\n B. Lake Pontchartrain\n C. The great lakes\n D. Niagara Falls\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 114, "native_id": "9-501", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 115, "native_id": "9-757", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 115, "native_id": "9-757", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 115, "native_id": "9-757", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\n A. old\n B. dead\n C. in an egg\n D. big\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 115, "native_id": "9-757", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 116, "native_id": "7-725", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 116, "native_id": "7-725", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 116, "native_id": "7-725", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. basements to flood due to weather\n B. small birds to kill large birds\n C. waterfalls to flow backwards\n D. stones to weather down to pebbles\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 116, "native_id": "7-725", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 117, "native_id": "1300", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 117, "native_id": "1300", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 117, "native_id": "1300", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Inherited characteristics\n A. include mice being able to navigate a maze\n B. include learning to sit on command\n C. include dolphins doing tricks for their trainers\n D. include spots on a ladybug\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 117, "native_id": "1300", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 118, "native_id": "9-230", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 118, "native_id": "9-230", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 118, "native_id": "9-230", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How could we determine approximately how far a bird is from the ground?\n A. Measure the altitude of the bird using a reference point, such as a tall building.\n B. Identify the species of bird\n C. Ask the bird how high it was when it returns back to earth\n D. Measure the bird's mass\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 118, "native_id": "9-230", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 119, "native_id": "9-988", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 119, "native_id": "9-988", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 119, "native_id": "9-988", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Endangered pandas are sometimes\n A. accidentally dropped into volcanoes\n B. confined to enclosures to be viewed by the public\n C. found eating corn in the middle of North America\n D. made into delicious rare steaks\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 119, "native_id": "9-988", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 120, "native_id": "9-393", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 120, "native_id": "9-393", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 120, "native_id": "9-393", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Algae can be found in\n A. reservoir\n B. meat\n C. street\n D. tree\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 120, "native_id": "9-393", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 121, "native_id": "7-823", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 121, "native_id": "7-823", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 121, "native_id": "7-823", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\n A. a campfire toasts bread\n B. a microwave heats soup\n C. a fire burns paper\n D. a small oven works\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 121, "native_id": "7-823", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 122, "native_id": "9-24", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 122, "native_id": "9-24", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 122, "native_id": "9-24", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\n A. mimicking other things\n B. melting into sand\n C. creating new homes\n D. mocking other fish\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 122, "native_id": "9-24", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 123, "native_id": "570", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 123, "native_id": "570", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 123, "native_id": "570", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does someone do when creating music?\n A. hit a toy baseball with a bat\n B. shake a baby rattle with your hand\n C. bang the wall with your fist\n D. pluck the strings of a fingerboard with your fingers\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 123, "native_id": "570", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 124, "native_id": "9-124", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 124, "native_id": "9-124", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 124, "native_id": "9-124", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\n A. telescope views\n B. water level fluctuations\n C. animal\n D. plant harvesting\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 124, "native_id": "9-124", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 125, "native_id": "9-199", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 125, "native_id": "9-199", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 125, "native_id": "9-199", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\n A. baby oil\n B. corn syrup or\n C. milk\n D. honey\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 125, "native_id": "9-199", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 126, "native_id": "767", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 126, "native_id": "767", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 126, "native_id": "767", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis can be performed by\n A. a cabbage cell\n B. a bee cell\n C. a bear cell\n D. a cat cell\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 126, "native_id": "767", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 127, "native_id": "28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 127, "native_id": "28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 127, "native_id": "28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\n A. reverse\n B. inverse\n C. equal\n D. direct\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 127, "native_id": "28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 128, "native_id": "9-1134", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 128, "native_id": "9-1134", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 128, "native_id": "9-1134", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: all cells use cellular respiration to\n A. photosynthesize\n B. release waste\n C. perform meiosis\n D. release energy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 128, "native_id": "9-1134", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 129, "native_id": "9-1030", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 129, "native_id": "9-1030", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 129, "native_id": "9-1030", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\n A. reproductive purposes\n B. viewing sounds\n C. sensing views\n D. sensing tastes\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 129, "native_id": "9-1030", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 130, "native_id": "9-18", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 130, "native_id": "9-18", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 130, "native_id": "9-18", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\n A. breaks for birds on the road\n B. given a large breakfast\n C. eats a few corn cobs\n D. given two apples to watch\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 130, "native_id": "9-18", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 131, "native_id": "8-378", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 131, "native_id": "8-378", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 131, "native_id": "8-378", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\n A. the woman makes more money\n B. the woman makes the same amount of money\n C. the woman spends more money\n D. the woman makes less money\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 131, "native_id": "8-378", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 132, "native_id": "7-677", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 132, "native_id": "7-677", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 132, "native_id": "7-677", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A desert environment is\n A. dry, grass covered, and humid\n B. lush, green, and tropical\n C. arid, parched, and sun-baked\n D. dry, damp, and lush\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 132, "native_id": "7-677", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 133, "native_id": "9-786", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 133, "native_id": "9-786", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 133, "native_id": "9-786", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seasons are caused by what rotating on its axis?\n A. Our Planet\n B. The Atmosphere\n C. The Equator\n D. The Sun\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 133, "native_id": "9-786", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 134, "native_id": "9-463", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 134, "native_id": "9-463", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 134, "native_id": "9-463", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is best an letting electricity pass through?\n A. tile flooring\n B. human flesh\n C. hockey stick\n D. a steak knife\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 134, "native_id": "9-463", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 135, "native_id": "7-71", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 135, "native_id": "7-71", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 135, "native_id": "7-71", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Asteroids crashing on planets can leave behind\n A. large, bowl-shaped cavities in the ground\n B. aliens and foreign foods\n C. small dents in the planet's core\n D. lakes filled with salty water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 135, "native_id": "7-71", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 136, "native_id": "9-1053", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 136, "native_id": "9-1053", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 136, "native_id": "9-1053", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the formula of the substance which best helps plants grow\n A. NH4\n B. C4H4\n C. CO2\n D. H2O\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 136, "native_id": "9-1053", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 137, "native_id": "9-437", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 137, "native_id": "9-437", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 137, "native_id": "9-437", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: You can experience a change of pressure when\n A. Yelling really loud\n B. Soaring the skies\n C. Going walking\n D. riding a bike\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 137, "native_id": "9-437", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 138, "native_id": "1787", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 138, "native_id": "1787", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 138, "native_id": "1787", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small creek absorbing heat energy can result in\n A. the creek water getting colder\n B. a parched creek bed\n C. tributaries branching off from the creek\n D. a runoff of extra water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 138, "native_id": "1787", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 139, "native_id": "7-107", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 139, "native_id": "7-107", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 139, "native_id": "7-107", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Summertime  happens during June in all but which location?\n A. Australia\n B. in Canada\n C. United States\n D. Europe\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 139, "native_id": "7-107", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 140, "native_id": "769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 140, "native_id": "769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 140, "native_id": "769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: It takes more water to fill a bathtub than a\n A. lake\n B. pool\n C. stomach\n D. holding tank\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 140, "native_id": "769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 141, "native_id": "9-73", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 141, "native_id": "9-73", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 141, "native_id": "9-73", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The eighth month of the year is winter in\n A. Brazil\n B. Indiana\n C. London\n D. Canada\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 141, "native_id": "9-73", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 142, "native_id": "9-1194", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 142, "native_id": "9-1194", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 142, "native_id": "9-1194", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: birds use their peckers to catch\n A. dogs\n B. a tan\n C. a ball\n D. bees\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 142, "native_id": "9-1194", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 143, "native_id": "9-416", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 143, "native_id": "9-416", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 143, "native_id": "9-416", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: transplanting seedling oaks has a positive impact on\n A. fuel costs\n B. the economy\n C. housing value\n D. the environment\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 143, "native_id": "9-416", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 144, "native_id": "470", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 144, "native_id": "470", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 144, "native_id": "470", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of hunting?\n A. humans throwing a spear through an animal\n B. humans chewing on boiled animal muscles\n C. humans gathering animals in a gate\n D. humans plucking fruit from a tree\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 144, "native_id": "470", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 145, "native_id": "1297", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 145, "native_id": "1297", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 145, "native_id": "1297", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\n A. paper\n B. wood\n C. stone\n D. orange juice\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 145, "native_id": "1297", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 146, "native_id": "8-346", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 146, "native_id": "8-346", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 146, "native_id": "8-346", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\n A. Sweet\n B. Salty\n C. bitter\n D. sour\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 146, "native_id": "8-346", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 147, "native_id": "7-807", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 147, "native_id": "7-807", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 147, "native_id": "7-807", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth's four layers are comprised mainly of\n A. stone\n B. bacteria\n C. water\n D. air\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 147, "native_id": "7-807", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 148, "native_id": "8-463", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 148, "native_id": "8-463", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 148, "native_id": "8-463", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\n A. the freezing point\n B. the prime point\n C. the boiling point\n D. the melting point\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 148, "native_id": "8-463", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 149, "native_id": "9-110", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 149, "native_id": "9-110", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 149, "native_id": "9-110", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Mammals give birth to live\n A. children\n B. birds\n C. fish\n D. insects\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 149, "native_id": "9-110", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 150, "native_id": "1611", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 150, "native_id": "1611", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 150, "native_id": "1611", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\n A. it loses its magnetization\n B. it loses its charge\n C. it may become magnetized\n D. it gains a charge\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 150, "native_id": "1611", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 151, "native_id": "9-942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 151, "native_id": "9-942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 151, "native_id": "9-942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\n A. luck\n B. humans\n C. gold\n D. inheritance\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 151, "native_id": "9-942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 152, "native_id": "9-1102", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 152, "native_id": "9-1102", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 152, "native_id": "9-1102", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which is recyclable?\n A. An Elephant\n B. A school notebook\n C. A boat\n D. A lake\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 152, "native_id": "9-1102", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 153, "native_id": "9-774", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 153, "native_id": "9-774", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 153, "native_id": "9-774", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\n A. it is a place to emigrate from\n B. it is an ideal habitat\n C. it is an unsustainable habitat\n D. it is a thriving abode\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 153, "native_id": "9-774", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 154, "native_id": "8-333", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 154, "native_id": "8-333", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 154, "native_id": "8-333", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\n A. water and soda\n B. water and oil\n C. sand and rocks\n D. salt and bark\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 154, "native_id": "8-333", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 155, "native_id": "9-573", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 155, "native_id": "9-573", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 155, "native_id": "9-573", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\n A. spider bites\n B. vomiting\n C. placebo effect\n D. dance fever\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 155, "native_id": "9-573", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 156, "native_id": "1955", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 156, "native_id": "1955", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 156, "native_id": "1955", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Lightning may lead to\n A. damage to local foliage\n B. rainbows across the sky\n C. growth of local flora\n D. firefighters getting the night off\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 156, "native_id": "1955", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 157, "native_id": "8-45", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 157, "native_id": "8-45", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 157, "native_id": "8-45", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When would a nocturnal predator most likely hunt?\n A. 5 p.m.\n B. 12 p.m.\n C. 3 a.m.\n D. 10 a.m.\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 157, "native_id": "8-45", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 158, "native_id": "9-674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 158, "native_id": "9-674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 158, "native_id": "9-674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where water be located in its gas form?\n A. inside a disc golf driver\n B. inside of a brass pipe\n C. a mile up in the sky\n D. inside a leather baseball\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 158, "native_id": "9-674", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 159, "native_id": "898", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 159, "native_id": "898", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 159, "native_id": "898", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Green parts of a life form absorb\n A. carbon dioxide\n B. light\n C. oxygen\n D. water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 159, "native_id": "898", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 160, "native_id": "7-1159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 160, "native_id": "7-1159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 160, "native_id": "7-1159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a new species of predator joins a community\n A. the new species will become herbivores\n B. prey will experience an increase in population\n C. prey will experience a drop in population\n D. the old species will die out\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 160, "native_id": "7-1159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 161, "native_id": "568", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 161, "native_id": "568", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 161, "native_id": "568", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\n A. It's dead\n B. For water\n C. For food\n D. For sun\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 161, "native_id": "568", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 162, "native_id": "9-877", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 162, "native_id": "9-877", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 162, "native_id": "9-877", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\n A. weasel\n B. snow fox\n C. wolf\n D. polar bear\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 162, "native_id": "9-877", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 163, "native_id": "406", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 163, "native_id": "406", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 163, "native_id": "406", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dairy is a source of\n A. a vitamin that prevents blood loss\n B. a vitamin that treats amino acid deficiency\n C. a group of fat-soluble secosteroids\n D. a vitamin that helps treat liver problems\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 163, "native_id": "406", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 164, "native_id": "7-1132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 164, "native_id": "7-1132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 164, "native_id": "7-1132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Atomic 26 is drawn to a device, it could be\n A. magnetized\n B. Na\n C. compass\n D. K\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 164, "native_id": "7-1132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 165, "native_id": "7-479", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 165, "native_id": "7-479", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 165, "native_id": "7-479", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: why do tadpoles change into frogs?\n A. tadpoles change to different animals\n B. tadpoles are really just fish\n C. they are young frogs still growing\n D. none of these\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 165, "native_id": "7-479", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 166, "native_id": "609", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 166, "native_id": "609", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 166, "native_id": "609", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do cows eat?\n A. Chickpeas\n B. Chocolate\n C. Steak\n D. Poultry\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 166, "native_id": "609", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 167, "native_id": "1568", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 167, "native_id": "1568", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 167, "native_id": "1568", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which object conducts electricity?\n A. Window\n B. Rubik's Cube\n C. Ship Anchor\n D. Boulder\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 167, "native_id": "1568", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 168, "native_id": "9-418", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 168, "native_id": "9-418", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 168, "native_id": "9-418", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Glucose travels\n A. from roots to leaves of a daffodil\n B. from a rose's leaves to the atmosphere\n C. from a daisy's leaves into it's underground support system\n D. from the sun to a sunflower's buds\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 168, "native_id": "9-418", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 169, "native_id": "7-1050", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 169, "native_id": "7-1050", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 169, "native_id": "7-1050", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. only happens in the summer\n B. is like nature's disappearing water trick\n C. is caused by snow\n D. involves the disappearance of sunlight\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 169, "native_id": "7-1050", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 170, "native_id": "9-510", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 170, "native_id": "9-510", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 170, "native_id": "9-510", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\n A. air\n B. Corn\n C. Runoff\n D. farmers\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 170, "native_id": "9-510", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 171, "native_id": "9-519", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 171, "native_id": "9-519", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 171, "native_id": "9-519", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\n A. friction\n B. metal\n C. wishing\n D. magic\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 171, "native_id": "9-519", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 172, "native_id": "9-637", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 172, "native_id": "9-637", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 172, "native_id": "9-637", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a duck like to live?\n A. the Sahara\n B. Antarctica\n C. the Appalachian mountains\n D. Death Valley\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 172, "native_id": "9-637", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 173, "native_id": "473", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 173, "native_id": "473", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 173, "native_id": "473", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What do tuna eat?\n A. Atlantic menhaden\n B. Swedish fish\n C. gummy fish\n D. laminariales\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 173, "native_id": "473", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 174, "native_id": "8-445", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 174, "native_id": "8-445", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 174, "native_id": "8-445", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\n A. flowers blooming\n B. grass turning brown\n C. trees growing\n D. blossoms blooming\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 174, "native_id": "8-445", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 175, "native_id": "9-575", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 175, "native_id": "9-575", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 175, "native_id": "9-575", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: is it normal for an adult animal to lay eggs?\n A. it has never happened\n B. yes it is standard\n C. it is abnormal and weird\n D. all of these\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 175, "native_id": "9-575", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 176, "native_id": "7-284", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 176, "native_id": "7-284", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 176, "native_id": "7-284", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog is overweight\n A. add more fat to their diet\n B. cut back their caloric intake\n C. let them sleep more\n D. increase their caloric intake\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 176, "native_id": "7-284", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 177, "native_id": "8-135", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 177, "native_id": "8-135", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 177, "native_id": "8-135", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Eyes allow humans\n A. to detect when a traffic light changes\n B. detect sour flavors in candy\n C. hear music at concerts\n D. detect acrid odors in the air\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 177, "native_id": "8-135", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 178, "native_id": "397", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 178, "native_id": "397", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 178, "native_id": "397", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Organisms covered by layers of sediment\n A. become fossils over night\n B. may end up reanimated over time\n C. develop characteristics for survival\n D. may end up fueling a car\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 178, "native_id": "397", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 179, "native_id": "9-32", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 179, "native_id": "9-32", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 179, "native_id": "9-32", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The winter solstice is on December 21st in the\n A. counties\n B. north of equator\n C. states\n D. southern hemisphere\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 179, "native_id": "9-32", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 180, "native_id": "48", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 180, "native_id": "48", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 180, "native_id": "48", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\n A. bleaching\n B. burning\n C. fossilization\n D. drying\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 180, "native_id": "48", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 181, "native_id": "8-69", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 181, "native_id": "8-69", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 181, "native_id": "8-69", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: were there fossil fuels in the ground when humans evolved?\n A. this was only created by humans\n B. humans predate fossil fuel formation\n C. significant supplies accumulated prior\n D. none of these\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 181, "native_id": "8-69", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 182, "native_id": "9-159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 182, "native_id": "9-159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 182, "native_id": "9-159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\n A. some water is around it\n B. water rained on it\n C. natural waters weathered it\n D. a pool was opened\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 182, "native_id": "9-159", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 183, "native_id": "9-317", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 183, "native_id": "9-317", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 183, "native_id": "9-317", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What type of useful product can be made from the moving winds?\n A. wood\n B. bananas\n C. electricity\n D. metal\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 183, "native_id": "9-317", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 184, "native_id": "423", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 184, "native_id": "423", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 184, "native_id": "423", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sides of the canyon are\n A. metal\n B. water\n C. rivers\n D. stone\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 184, "native_id": "423", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 185, "native_id": "8-304", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 185, "native_id": "8-304", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 185, "native_id": "8-304", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these people would have the worst air quality at their residence?\n A. a man who lives next to a landfill\n B. a man who lives in a city with the best air quality\n C. none of these\n D. a man who lives in a great suburb\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 185, "native_id": "8-304", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 186, "native_id": "785", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 186, "native_id": "785", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 186, "native_id": "785", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is a stopwatch used for?\n A. to rewind 5 minutes\n B. to tell what will happen 5 minutes from now\n C. to voice the time\n D. to measure minutes and hours\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 186, "native_id": "785", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 187, "native_id": "9-1087", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 187, "native_id": "9-1087", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 187, "native_id": "9-1087", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: As a drought worsens the level at an aquifer will\n A. stay the same\n B. fluctuate wildly\n C. decrease\n D. increase\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 187, "native_id": "9-1087", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 188, "native_id": "485", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 188, "native_id": "485", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 188, "native_id": "485", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind frequently helps transport from one place to another\n A. marble statues\n B. molten magma\n C. subterranean termites\n D. exposed topsoil\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 188, "native_id": "485", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 189, "native_id": "9-908", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 189, "native_id": "9-908", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 189, "native_id": "9-908", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\n A. bike helmet\n B. avoiding accidents\n C. friction\n D. gearing\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 189, "native_id": "9-908", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 190, "native_id": "1231", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 190, "native_id": "1231", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 190, "native_id": "1231", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of spices have\n A. crystals\n B. feathers\n C. cell walls\n D. craters\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 190, "native_id": "1231", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 191, "native_id": "810", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 191, "native_id": "810", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 191, "native_id": "810", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Conservation\n A. leads to longer drought of resources\n B. leads to longer availability of resources\n C. leads to more consumption\n D. leads to short supply of resources\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 191, "native_id": "810", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 192, "native_id": "158", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 192, "native_id": "158", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 192, "native_id": "158", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A tree is not the habitat of a\n A. squirrel\n B. woodpecker\n C. monkey\n D. lion\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 192, "native_id": "158", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 193, "native_id": "7-445", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 193, "native_id": "7-445", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 193, "native_id": "7-445", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion could lead to\n A. a change in the direction of a stream\n B. a change in ocean temperatures\n C. an increase in rainy weather\n D. an increase in plants and animals\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 193, "native_id": "7-445", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 194, "native_id": "1502", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 194, "native_id": "1502", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 194, "native_id": "1502", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A hemisphere experiences summer when\n A. it's tilted towards Jupiter\n B. it's angled towards the moon\n C. it's angled towards the largest star in the solar system\n D. it spins counter clockwise on Earth's axis\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 194, "native_id": "1502", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 195, "native_id": "1200", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 195, "native_id": "1200", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 195, "native_id": "1200", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What date is the amount of daylight minimized\n A. Jul 4th\n B. Feb 29th\n C. May 3rd\n D. Sep 1st\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 195, "native_id": "1200", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 196, "native_id": "437", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 196, "native_id": "437", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 196, "native_id": "437", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which term is involved with protection by skin?\n A. Eucerin pH5 range\n B. Sagittal plane\n C. pyogenic vibrio\n D. popliteus\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 196, "native_id": "437", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 197, "native_id": "8-205", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 197, "native_id": "8-205", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 197, "native_id": "8-205", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The reason Earth is so sturdy is because\n A. It is made from rock\n B. It eats three meals a day\n C. It has a loving family\n D. It is made from metal\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 197, "native_id": "8-205", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 198, "native_id": "9-270", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 198, "native_id": "9-270", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 198, "native_id": "9-270", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant left in the dark\n A. produces fruit\n B. grows faster\n C. fails to grow\n D. gets greener\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 198, "native_id": "9-270", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 199, "native_id": "8-130", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 199, "native_id": "8-130", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 199, "native_id": "8-130", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\n A. heat\n B. metal\n C. lithium-ion\n D. plastic\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 199, "native_id": "8-130", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 200, "native_id": "229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 200, "native_id": "229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 200, "native_id": "229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\n A. axial planetary rotation\n B. comets\n C. planetary formation\n D. the sun's rays\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 200, "native_id": "229", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 201, "native_id": "9-390", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 201, "native_id": "9-390", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 201, "native_id": "9-390", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which source provides the safest water?\n A. River\n B. Sea\n C. Ocean\n D. Rain\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 201, "native_id": "9-390", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 202, "native_id": "8-107", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 202, "native_id": "8-107", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 202, "native_id": "8-107", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\n A. spread out some fur\n B. has black on his ears\n C. passed down inherited characteristics\n D. is the same size as the mother\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 202, "native_id": "8-107", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 203, "native_id": "7-527", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 203, "native_id": "7-527", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 203, "native_id": "7-527", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis means plants are unable to\n A. convert sunlight to sand\n B. provide food sources for others\n C. be producers in an ecosystem\n D. make their own food\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 203, "native_id": "7-527", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 204, "native_id": "7-333", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 204, "native_id": "7-333", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 204, "native_id": "7-333", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Through DNA, a rabbit will have long ears if\n A. rabbits are born with ears\n B. there was a lot of food\n C. genetic contributors had long ears\n D. parents were also rabbits\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 204, "native_id": "7-333", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 205, "native_id": "9-44", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 205, "native_id": "9-44", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 205, "native_id": "9-44", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are various creatures that live in forests, such as\n A. giant fish\n B. enormous crabs\n C. whitetails\n D. desert jackals\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 205, "native_id": "9-44", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 206, "native_id": "7-160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 206, "native_id": "7-160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 206, "native_id": "7-160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An octopus protects itself with\n A. water splashing\n B. running fast\n C. long hands\n D. pigment squirting\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 206, "native_id": "7-160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 207, "native_id": "1942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 207, "native_id": "1942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 207, "native_id": "1942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Resources decreasing in an environment\n A. induces organisms to use more of their resources\n B. causes an increase in use of resources\n C. causes an uptick in birthrate\n D. induces organisms to be more economical with resources\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 207, "native_id": "1942", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 208, "native_id": "9-597", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 208, "native_id": "9-597", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 208, "native_id": "9-597", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\n A. insects\n B. weathering\n C. lava\n D. erosion\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 208, "native_id": "9-597", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 209, "native_id": "9-35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 209, "native_id": "9-35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 209, "native_id": "9-35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\n A. millions of years\n B. 1 day\n C. 10 days\n D. 100 days\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 209, "native_id": "9-35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 210, "native_id": "1161", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 210, "native_id": "1161", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 210, "native_id": "1161", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Moon phases\n A. change the moon into cheese\n B. alter the way the moon's facade looks\n C. change moon lakes into vapor\n D. cause lunar eclipse every day\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 210, "native_id": "1161", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 211, "native_id": "7-171", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 211, "native_id": "7-171", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 211, "native_id": "7-171", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\n A. they would be scalded\n B. organs would remain uneffected\n C. they would begin to decay\n D. they would experience pain\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 211, "native_id": "7-171", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 212, "native_id": "1139", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 212, "native_id": "1139", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 212, "native_id": "1139", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Barnyard bovines\n A. eat organic chicken\n B. eat eggs\n C. eat beef\n D. eat alfalfa hay\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 212, "native_id": "1139", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 213, "native_id": "1924", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 213, "native_id": "1924", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 213, "native_id": "1924", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments features\n A. tropical plants\n B. tons of sun\n C. massive rain totals\n D. icy precipitation\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 213, "native_id": "1924", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 214, "native_id": "9-440", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 214, "native_id": "9-440", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 214, "native_id": "9-440", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\n A. army of ants\n B. a community\n C. a toy store\n D. a shopping mall\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 214, "native_id": "9-440", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 215, "native_id": "9-528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 215, "native_id": "9-528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 215, "native_id": "9-528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\n A. the lungs\n B. the kidney\n C. the heart\n D. the liver\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 215, "native_id": "9-528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 216, "native_id": "170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 216, "native_id": "170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 216, "native_id": "170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which pair don't reproduce the same way?\n A. rabbit and hare\n B. mule and hinny\n C. cat and catfish\n D. caterpillar and butterfly\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 216, "native_id": "170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 217, "native_id": "395", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 217, "native_id": "395", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 217, "native_id": "395", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is warm blooded?\n A. toad\n B. snake\n C. turtle\n D. skunk\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 217, "native_id": "395", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 218, "native_id": "9-633", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 218, "native_id": "9-633", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 218, "native_id": "9-633", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\n A. a manager\n B. an agent\n C. a meal\n D. a reproductive companion\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 218, "native_id": "9-633", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 219, "native_id": "9-504", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 219, "native_id": "9-504", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 219, "native_id": "9-504", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Vast quantities of metal can be obtained from\n A. a quarry\n B. concerts\n C. forests\n D. salt\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 219, "native_id": "9-504", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 220, "native_id": "8-192", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 220, "native_id": "8-192", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 220, "native_id": "8-192", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Roasting a turkey requires adding what type of energy\n A. Heat\n B. Kinetic\n C. Magnetic\n D. Chemical\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 220, "native_id": "8-192", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 221, "native_id": "7-1108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 221, "native_id": "7-1108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 221, "native_id": "7-1108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dry environments often\n A. liberally use water for everything\n B. allow plants to flourish\n C. require people to move\n D. institute rules about water usage\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 221, "native_id": "7-1108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 222, "native_id": "7-852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 222, "native_id": "7-852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 222, "native_id": "7-852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Phloem moves things around a plant similar to how\n A. blood moves in a body\n B. leaves move in the wind\n C. water moves in a system\n D. cars move on a street\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 222, "native_id": "7-852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 223, "native_id": "761", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 223, "native_id": "761", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 223, "native_id": "761", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would a polar bear be most comfortable?\n A. Arizona\n B. Georgia\n C. Florida\n D. Nebraska\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 223, "native_id": "761", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 224, "native_id": "8-318", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 224, "native_id": "8-318", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 224, "native_id": "8-318", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\n A. light is far away\n B. light is more easily seen\n C. light is more distant\n D. light is further away\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 224, "native_id": "8-318", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 225, "native_id": "636", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 225, "native_id": "636", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 225, "native_id": "636", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Members of rock bands often perform with\n A. flutes\n B. sandals\n C. earplugs\n D. gloves\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 225, "native_id": "636", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 226, "native_id": "7-444", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 226, "native_id": "7-444", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 226, "native_id": "7-444", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Muscles move bones to produce movement like when\n A. arms are resting\n B. hair is growing\n C. smiles are invisible\n D. toes are wiggled\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 226, "native_id": "7-444", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 227, "native_id": "8-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 227, "native_id": "8-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 227, "native_id": "8-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How is electricity produced from the ocean?\n A. decaying organic material from sealife\n B. energy is accessed underwater from tides\n C. drills to access oil supplies\n D. chemical reactions produced from the salt in the water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 227, "native_id": "8-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 228, "native_id": "9-187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 228, "native_id": "9-187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 228, "native_id": "9-187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: small reptile's diet consists mostly of\n A. invertebrates\n B. insects\n C. mammals\n D. fish\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 228, "native_id": "9-187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 229, "native_id": "1345", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 229, "native_id": "1345", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 229, "native_id": "1345", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The life work of a flower is to\n A. provide nice scents\n B. be successfully fertilized\n C. grow very tall\n D. look pretty\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 229, "native_id": "1345", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 230, "native_id": "8-59", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 230, "native_id": "8-59", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 230, "native_id": "8-59", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\n A. the plastic water bottle\n B. the backside of a person\n C. the hair on a doll\n D. the chassis of a car\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 230, "native_id": "8-59", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 231, "native_id": "178", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 231, "native_id": "178", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 231, "native_id": "178", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: During landslides there is often a lot of\n A. air\n B. mud\n C. snow\n D. wind\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 231, "native_id": "178", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 232, "native_id": "9-1186", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 232, "native_id": "9-1186", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 232, "native_id": "9-1186", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of a chemical reaction would be\n A. A rusty fence\n B. Sleeping\n C. Drinking water\n D. Rain\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 232, "native_id": "9-1186", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 233, "native_id": "82", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 233, "native_id": "82", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 233, "native_id": "82", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\n A. equal\n B. inverse\n C. direct\n D. reverse\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 233, "native_id": "82", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 234, "native_id": "8-165", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 234, "native_id": "8-165", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 234, "native_id": "8-165", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tuna primarily eat\n A. parasites, soybeans and flaxseeds\n B. sea turtles, sharks and coral reefs\n C. spineless marine organisms, cartilaginous and gelatinous organisms\n D. sea vegetables like kelp, Irish moss and Arame\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 234, "native_id": "8-165", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 235, "native_id": "404", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 235, "native_id": "404", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 235, "native_id": "404", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation of water can lead to\n A. waterfalls\n B. blizzards\n C. earthquakes\n D. hot springs\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 235, "native_id": "404", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 236, "native_id": "279", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 236, "native_id": "279", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 236, "native_id": "279", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an electrical energy conductor?\n A. horseshoe\n B. tire\n C. cotton shirt\n D. maple branch\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 236, "native_id": "279", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 237, "native_id": "9-532", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 237, "native_id": "9-532", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 237, "native_id": "9-532", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of substance will cool when it touches a cold object?\n A. warm\n B. frozen\n C. chilly\n D. cold\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 237, "native_id": "9-532", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 238, "native_id": "268", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 238, "native_id": "268", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 238, "native_id": "268", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Scraping an object\n A. may cause the object to grow in size\n B. may cause the object to fall\n C. may cause pieces to flake off the object\n D. may cause the object to snap in half\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 238, "native_id": "268", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 239, "native_id": "7-1018", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 239, "native_id": "7-1018", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 239, "native_id": "7-1018", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pulley is used to lift a flag on a flagpole by\n A. moving a rope sideways\n B. putting something in the air\n C. moving things with wheels\n D. yanking string up a wheel\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 239, "native_id": "7-1018", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 240, "native_id": "1756", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 240, "native_id": "1756", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 240, "native_id": "1756", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal lays eggs\n A. emus\n B. dogs\n C. squirrels\n D. giraffes\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 240, "native_id": "1756", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 241, "native_id": "1137", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 241, "native_id": "1137", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 241, "native_id": "1137", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dunes can be made out of the same thing as\n A. clothes\n B. food\n C. forests\n D. castles\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 241, "native_id": "1137", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 242, "native_id": "7-203", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 242, "native_id": "7-203", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 242, "native_id": "7-203", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A learned behavior is exhibited when\n A. squinting in bright light\n B. inhaling and exhaling during sleep\n C. blinking and gulping air\n D. nailing up a picture frame\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 242, "native_id": "7-203", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 243, "native_id": "745", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 243, "native_id": "745", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 243, "native_id": "745", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wax can be used similarly to\n A. wood\n B. rubber\n C. water\n D. metal\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 243, "native_id": "745", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 244, "native_id": "7-902", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 244, "native_id": "7-902", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 244, "native_id": "7-902", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\n A. get bigger yields\n B. make things boring\n C. keep things random\n D. get smaller yields\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 244, "native_id": "7-902", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 245, "native_id": "1095", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 245, "native_id": "1095", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 245, "native_id": "1095", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\n A. feathers slows heat transfer\n B. the natural duck wax\n C. a synthetic thick liner\n D. small flexible solar panels\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 245, "native_id": "1095", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 246, "native_id": "7-163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 246, "native_id": "7-163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 246, "native_id": "7-163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Turbines churning seawater can be used to produce what?\n A. a charge for appliances\n B. large quantities of soup\n C. large schools of fish\n D. creating some sharp cheese\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 246, "native_id": "7-163", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 247, "native_id": "9-858", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 247, "native_id": "9-858", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 247, "native_id": "9-858", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will increase when a substance absorbs solar energy?\n A. weight\n B. height\n C. hotness\n D. nutrition\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 247, "native_id": "9-858", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 248, "native_id": "1530", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 248, "native_id": "1530", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 248, "native_id": "1530", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these travels through the air?\n A. planets\n B. thoughts\n C. automobile\n D. music\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 248, "native_id": "1530", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 249, "native_id": "9-993", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 249, "native_id": "9-993", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 249, "native_id": "9-993", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A company makes notebooks for college courses, so their main material is\n A. chips\n B. water\n C. grass\n D. trees\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 249, "native_id": "9-993", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 250, "native_id": "8-340", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 250, "native_id": "8-340", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 250, "native_id": "8-340", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Rain is usually guaranteed when all are present but\n A. cirrus clouds\n B. cumulus clouds\n C. hail stones\n D. direct sunshine\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 250, "native_id": "8-340", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 251, "native_id": "3", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 251, "native_id": "3", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 251, "native_id": "3", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Sources of air pollution are\n A. Walking\n B. Landfills\n C. Water\n D. Chips\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 251, "native_id": "3", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 252, "native_id": "1074", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 252, "native_id": "1074", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 252, "native_id": "1074", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are iron nails made out of?\n A. wood\n B. plastic\n C. metal\n D. glass\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 252, "native_id": "1074", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 253, "native_id": "9-431", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 253, "native_id": "9-431", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 253, "native_id": "9-431", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\n A. combining\n B. drinking\n C. Subtracting\n D. throwing\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 253, "native_id": "9-431", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 254, "native_id": "9-638", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 254, "native_id": "9-638", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 254, "native_id": "9-638", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A moth leaving it's cocoon is the final step in a\n A. life cycle\n B. transformation\n C. recycling process\n D. chemical reaction\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 254, "native_id": "9-638", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 255, "native_id": "9-352", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 255, "native_id": "9-352", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 255, "native_id": "9-352", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\n A. ice and snow\n B. sticks and stones\n C. snow and water\n D. iron and carbon\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 255, "native_id": "9-352", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 256, "native_id": "226", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 256, "native_id": "226", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 256, "native_id": "226", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is powered the same way an electric car is?\n A. a bicycle\n B. a motorcycle\n C. a propane grill\n D. a blender\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 256, "native_id": "226", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 257, "native_id": "9-132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 257, "native_id": "9-132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 257, "native_id": "9-132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Aluminum is what?\n A. reprocessable\n B. plastic\n C. liquid\n D. absorbent\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 257, "native_id": "9-132", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 258, "native_id": "9-222", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 258, "native_id": "9-222", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 258, "native_id": "9-222", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To get warm frogs can\n A. wear a Christmas sweater\n B. Drink a hot chocolate\n C. Go for a run\n D. sit under a lamp\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 258, "native_id": "9-222", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 259, "native_id": "9-105", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 259, "native_id": "9-105", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 259, "native_id": "9-105", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\n A. it exhibits absolute rigidity\n B. it is a solid mass\n C. all of these\n D. it is a variable substance\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 259, "native_id": "9-105", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 260, "native_id": "7-459", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 260, "native_id": "7-459", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 260, "native_id": "7-459", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earthworms create tunnels in\n A. ice\n B. dirt\n C. water\n D. concrete\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 260, "native_id": "7-459", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 261, "native_id": "9-881", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 261, "native_id": "9-881", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 261, "native_id": "9-881", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\n A. regurgitation\n B. decomposition\n C. recycling\n D. burial\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 261, "native_id": "9-881", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 262, "native_id": "280", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 262, "native_id": "280", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 262, "native_id": "280", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The aluminum cans were much hotter than the\n A. gold jewelry\n B. wooden fence\n C. brass doorknob\n D. steel pole\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 262, "native_id": "280", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 263, "native_id": "187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 263, "native_id": "187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 263, "native_id": "187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following is not an input in photosynthesis?\n A. sunlight\n B. oxygen\n C. water\n D. carbon dioxide\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 263, "native_id": "187", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 264, "native_id": "8-253", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 264, "native_id": "8-253", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 264, "native_id": "8-253", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these are you most likely to find in a desert?\n A. a hammer head shark\n B. a big tilapia fish\n C. a prickly horned male lizard\n D. none of these\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 264, "native_id": "8-253", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 265, "native_id": "9-482", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 265, "native_id": "9-482", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 265, "native_id": "9-482", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\n A. it will be fine\n B. it will be happy\n C. it will be comfortable\n D. It will pass out\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 265, "native_id": "9-482", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 266, "native_id": "496", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 266, "native_id": "496", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 266, "native_id": "496", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following can be used to turn on an electrical device?\n A. solar-rechargeable battery\n B. a wedge\n C. a magnet\n D. pressure gauge\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 266, "native_id": "496", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 267, "native_id": "630", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 267, "native_id": "630", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 267, "native_id": "630", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following contains large amounts of salt water?\n A. The Amazon\n B. The Nile\n C. The Indian\n D. The Mississippi\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 267, "native_id": "630", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 268, "native_id": "9-16", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 268, "native_id": "9-16", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 268, "native_id": "9-16", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\n A. the butterfly\n B. the sloth\n C. the praying mantis\n D. the antelope\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 268, "native_id": "9-16", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 269, "native_id": "7-986", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 269, "native_id": "7-986", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 269, "native_id": "7-986", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person speaks English as her first language because\n A. media is mainly in English\n B. school is in English\n C. she was genetically predisposed\n D. she watched her parents speak\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 269, "native_id": "7-986", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 270, "native_id": "7-787", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 270, "native_id": "7-787", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 270, "native_id": "7-787", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what are eaten by honey producing insects?\n A. they consume plants\n B. they eat cows\n C. plant reproduction parts\n D. they eat flowers\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 270, "native_id": "7-787", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 271, "native_id": "9-181", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 271, "native_id": "9-181", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 271, "native_id": "9-181", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Earth's closest heat source is\n A. our celestial fireball\n B. solar flares\n C. gamma rays\n D. big bang\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 271, "native_id": "9-181", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 272, "native_id": "1240", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 272, "native_id": "1240", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 272, "native_id": "1240", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How can we see how wind effects sand?\n A. sand is always moving\n B. sandstorms create ripples in sand\n C. sand is easy to move through\n D. beaches often have waves in the sand\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 272, "native_id": "1240", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 273, "native_id": "474", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 273, "native_id": "474", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 273, "native_id": "474", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fruit comes from what source\n A. an organism that releases carbon dioxide\n B. an organism that absorbs water through it's branches\n C. an organism that absorbs oxygen\n D. an organism that absorbs water through it's roots\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 273, "native_id": "474", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 274, "native_id": "1274", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 274, "native_id": "1274", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 274, "native_id": "1274", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In which location would a groundhog hide from a wolf?\n A. beside a tree\n B. in the grass\n C. on a stump\n D. under the ground\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 274, "native_id": "1274", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 275, "native_id": "1531", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 275, "native_id": "1531", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 275, "native_id": "1531", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cutting down trees in a forest\n A. leads to more habitats for animals\n B. decreases the chance of erosion\n C. increases the number of trees in the forest\n D. leads to less habitats for animals\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 275, "native_id": "1531", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 276, "native_id": "8-321", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 276, "native_id": "8-321", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 276, "native_id": "8-321", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\n A. animals\n B. resistance\n C. water\n D. sunshine\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 276, "native_id": "8-321", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 277, "native_id": "1321", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 277, "native_id": "1321", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 277, "native_id": "1321", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The arctic is white in coloring\n A. because it's overpopulated with polar bears\n B. because it's covered in white lilies\n C. because it's blanketed in crystalline ice water\n D. because it's gets so little sunlight\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 277, "native_id": "1321", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 278, "native_id": "9-51", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 278, "native_id": "9-51", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 278, "native_id": "9-51", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\n A. Lots of meat\n B. Lots of love\n C. Vitamin supplements\n D. An insulated room\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 278, "native_id": "9-51", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 279, "native_id": "7-685", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 279, "native_id": "7-685", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 279, "native_id": "7-685", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The majority of a lizard's diet consists of\n A. fleas\n B. crawlies\n C. gummy worms\n D. berries\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 279, "native_id": "7-685", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 280, "native_id": "7-59", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 280, "native_id": "7-59", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 280, "native_id": "7-59", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What food production happens in a leaf?\n A. nutrient making process\n B. the breathing\n C. the respiration\n D. the digestion\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 280, "native_id": "7-59", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 281, "native_id": "7-270", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 281, "native_id": "7-270", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 281, "native_id": "7-270", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Plants are unable to grow if they have zero access to\n A. a nice cool breeze\n B. fresh soil with manure\n C. a regular source of saltwater\n D. needs required for creating chlorophyll\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 281, "native_id": "7-270", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 282, "native_id": "7-736", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 282, "native_id": "7-736", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 282, "native_id": "7-736", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which characteristic did a person inherit?\n A. length of hair\n B. number of friends\n C. number of nails\n D. length of shirt\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 282, "native_id": "7-736", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 283, "native_id": "8-186", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 283, "native_id": "8-186", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 283, "native_id": "8-186", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Selective deforestation has a negative impact on\n A. rain clouds and ozone layer\n B. lakes, ponds and shellfish\n C. greenhouse gases and algae\n D. living organisms in ecosystem\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 283, "native_id": "8-186", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 284, "native_id": "224", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 284, "native_id": "224", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 284, "native_id": "224", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Where would you find a mine?\n A. in a tree\n B. under a mountain\n C. in the air\n D. in the water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 284, "native_id": "224", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 285, "native_id": "8-206", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 285, "native_id": "8-206", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 285, "native_id": "8-206", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What can cause people to crash their car?\n A. Seeing a solar eclipse\n B. Using their turn signals\n C. Driving the speed limit\n D. Keeping their eyes on the road\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 285, "native_id": "8-206", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 286, "native_id": "8-190", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 286, "native_id": "8-190", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 286, "native_id": "8-190", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coral grows in\n A. frigid waters\n B. tepid seas\n C. glacial environments\n D. jungle forests\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 286, "native_id": "8-190", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 287, "native_id": "7-334", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 287, "native_id": "7-334", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 287, "native_id": "7-334", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\n A. certain things may be featured\n B. certain features may be predicted\n C. certain traits may be given\n D. certain features may be guaranteed\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 287, "native_id": "7-334", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 288, "native_id": "9-853", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 288, "native_id": "9-853", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 288, "native_id": "9-853", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\n A. longer antlers\n B. food for itself\n C. baby deer\n D. urine\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 288, "native_id": "9-853", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 289, "native_id": "8-367", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 289, "native_id": "8-367", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 289, "native_id": "8-367", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Building new areas to dispose of refuse may lead to\n A. community concerns over environmental impact\n B. better air and soil quality\n C. higher value on land parcels\n D. improvement in water supply\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 289, "native_id": "8-367", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 290, "native_id": "1047", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 290, "native_id": "1047", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 290, "native_id": "1047", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Evaporation\n A. causes puddles to become dried out mud\n B. causes fields of crops to grow faster\n C. causes flowers to bloom abundantly\n D. fills up irrigation ponds\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 290, "native_id": "1047", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 291, "native_id": "9-454", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 291, "native_id": "9-454", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 291, "native_id": "9-454", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\n A. pythons\n B. salmon\n C. robins\n D. craters\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 291, "native_id": "9-454", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 292, "native_id": "1572", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 292, "native_id": "1572", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 292, "native_id": "1572", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item urinates?\n A. airplane\n B. car\n C. mammal\n D. boat\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 292, "native_id": "1572", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 293, "native_id": "8-373", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 293, "native_id": "8-373", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 293, "native_id": "8-373", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: All of the following are examples of evaporation apart from\n A. Warm breath fogging up a mirror\n B. Morning dew drying on the grass\n C. The water level in a glass decreasing\n D. Sweat drying on skin\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 293, "native_id": "8-373", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 294, "native_id": "9-772", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 294, "native_id": "9-772", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 294, "native_id": "9-772", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The lunar cycle also changes\n A. water\n B. colors\n C. the sun\n D. planets\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 294, "native_id": "9-772", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 295, "native_id": "1852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 295, "native_id": "1852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 295, "native_id": "1852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An organism that makes food for itself\n A. is nutritionally self sustaining\n B. will die faster than other organisms\n C. will need help sustaining strength\n D. is reliant on other organisms for assistance\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 295, "native_id": "1852", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 296, "native_id": "9-1090", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 296, "native_id": "9-1090", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 296, "native_id": "9-1090", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What does the respiratory system transfer to the circulatory system?\n A. food\n B. water\n C. nutrients\n D. O\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 296, "native_id": "9-1090", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 297, "native_id": "7-769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 297, "native_id": "7-769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 297, "native_id": "7-769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a closed circuit, electricity will\n A. burn out\n B. charge itself\n C. loop endlessly\n D. resist flow\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 297, "native_id": "7-769", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 298, "native_id": "9-478", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 298, "native_id": "9-478", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 298, "native_id": "9-478", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\n A. mother\n B. grandfather\n C. daughter\n D. father\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 298, "native_id": "9-478", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 299, "native_id": "448", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 299, "native_id": "448", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 299, "native_id": "448", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\n A. richer\n B. dryer\n C. sandy\n D. harder\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 299, "native_id": "448", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 300, "native_id": "7-417", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 300, "native_id": "7-417", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 300, "native_id": "7-417", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what kind of temperature causes fur shedding?\n A. in freezing cold\n B. a high temperature\n C. in any temperature\n D. a low temperature\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 300, "native_id": "7-417", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 301, "native_id": "7-108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 301, "native_id": "7-108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 301, "native_id": "7-108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: exposure to fire could result in\n A. wet items\n B. cold items\n C. none of these\n D. combusted items\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 301, "native_id": "7-108", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 302, "native_id": "1506", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 302, "native_id": "1506", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 302, "native_id": "1506", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of clear weather meaning sunny weather?\n A. more stars are visible on clear nights\n B. cloud cover protects from sunburn\n C. clear days will be warmer\n D. fewer clouds allow for more sunlight\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 302, "native_id": "1506", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 303, "native_id": "1712", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 303, "native_id": "1712", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 303, "native_id": "1712", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\n A. a wick\n B. a funnel\n C. a knife\n D. a whisk\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 303, "native_id": "1712", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 304, "native_id": "8-312", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 304, "native_id": "8-312", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 304, "native_id": "8-312", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a compass is a kind of tool for determining direction by pointing\n A. to western Canada shoreline\n B. to the lower pole\n C. to the upper pole\n D. directly to the equator\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 304, "native_id": "8-312", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 305, "native_id": "9-776", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 305, "native_id": "9-776", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 305, "native_id": "9-776", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these saws will last longer?\n A. iron saw\n B. aluminium saw\n C. plastic saw\n D. wooden saw\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 305, "native_id": "9-776", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 306, "native_id": "8-279", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 306, "native_id": "8-279", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 306, "native_id": "8-279", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\n A. the single moon close to us\n B. the orion star cluster\n C. the sun that shines all day\n D. all of these\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 306, "native_id": "8-279", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 307, "native_id": "9-621", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 307, "native_id": "9-621", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 307, "native_id": "9-621", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The moon is known for having what feature?\n A. frozen streams of water\n B. large bowl shaped cavities\n C. caves formed by solar winds\n D. groups of large trees\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 307, "native_id": "9-621", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 308, "native_id": "1823", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 308, "native_id": "1823", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 308, "native_id": "1823", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cellular respiration's trash is\n A. a bug's treasure\n B. a cow's treasure\n C. a plant's treasure\n D. everyone's trash\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 308, "native_id": "1823", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 309, "native_id": "9-735", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 309, "native_id": "9-735", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 309, "native_id": "9-735", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\n A. swimming in a lake\n B. building a new subdivision\n C. dancing in a field\n D. going for a hike\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 309, "native_id": "9-735", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 310, "native_id": "7-1170", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 310, "native_id": "7-1170", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 310, "native_id": "7-1170", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bird that takes off flying is\n A. using heat to produce motion\n B. using calories to produce motion\n C. using wings to produce heat\n D. using calories to produce energy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 310, "native_id": "7-1170", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 311, "native_id": "1500", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 311, "native_id": "1500", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 311, "native_id": "1500", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The leading cause of soil and rock erosion is\n A. H2O\n B. CO2\n C. NaCl\n D. Fe\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 311, "native_id": "1500", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 312, "native_id": "342", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 312, "native_id": "342", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 312, "native_id": "342", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these foods might have a negative impact on humans?\n A. Organic corn\n B. Conventional corn\n C. Organic potato\n D. Organic Apples\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 312, "native_id": "342", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 313, "native_id": "7-356", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 313, "native_id": "7-356", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 313, "native_id": "7-356", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of implement is a compass?\n A. to test heat\n B. for wind speed\n C. it measures distance\n D. it shows direction\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 313, "native_id": "7-356", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 314, "native_id": "78", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 314, "native_id": "78", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 314, "native_id": "78", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Nectar is taken to\n A. flowers\n B. a hive\n C. a stream\n D. a nest\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 314, "native_id": "78", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 315, "native_id": "9-520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 315, "native_id": "9-520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 315, "native_id": "9-520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what is the closest source of plasma to our planet?\n A. all of these\n B. the cloud in the sky\n C. the nearest star sulfur burning heavenly body\n D. the bare moon surface\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 315, "native_id": "9-520", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 316, "native_id": "7-653", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 316, "native_id": "7-653", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 316, "native_id": "7-653", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sailor needs to navigate to the shore, and does this by\n A. closing the sails quickly\n B. setting out to sea\n C. making an adjustment to the rudder\n D. taking the afternoon off\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 316, "native_id": "7-653", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 317, "native_id": "1112", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 317, "native_id": "1112", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 317, "native_id": "1112", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bat starts its life similarly to a\n A. chicken\n B. pig\n C. butterfly\n D. duck\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 317, "native_id": "1112", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 318, "native_id": "9-152", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 318, "native_id": "9-152", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 318, "native_id": "9-152", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order to catch a rabbit, a predator must be\n A. big\n B. quick\n C. slow\n D. small\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 318, "native_id": "9-152", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 319, "native_id": "9-552", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 319, "native_id": "9-552", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 319, "native_id": "9-552", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\n A. prey\n B. predator\n C. herbivore\n D. canary\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 319, "native_id": "9-552", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 320, "native_id": "7-262", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 320, "native_id": "7-262", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 320, "native_id": "7-262", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A warm-weather organism can be found in\n A. the Sahara\n B. the mountains\n C. the ocean\n D. the sewers\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 320, "native_id": "7-262", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 321, "native_id": "7-683", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 321, "native_id": "7-683", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 321, "native_id": "7-683", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When approaching an elephant from a great distance,\n A. it stays large\n B. it grows larger\n C. it gets bigger\n D. it looks bigger\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 321, "native_id": "7-683", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 322, "native_id": "276", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 322, "native_id": "276", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 322, "native_id": "276", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would cause a human to grow?\n A. light waves\n B. eating wheat\n C. photosynthesis\n D. marching\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 322, "native_id": "276", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 323, "native_id": "7-855", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 323, "native_id": "7-855", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 323, "native_id": "7-855", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A saguaro has adaptations for an environment with\n A. lots of snow\n B. many people\n C. less water\n D. more water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 323, "native_id": "7-855", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 324, "native_id": "664", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 324, "native_id": "664", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 324, "native_id": "664", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There are less hummingbirds by this house than before because of\n A. a feeder at this house\n B. the birds no longer like feeders\n C. the size of the feeder\n D. a feeder at another house\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 324, "native_id": "664", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 325, "native_id": "9-883", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 325, "native_id": "9-883", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 325, "native_id": "9-883", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the  oceans are full of\n A. water lilies\n B. guppies\n C. sea life\n D. fresh water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 325, "native_id": "9-883", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 326, "native_id": "9-550", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 326, "native_id": "9-550", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 326, "native_id": "9-550", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A light was off because the cord was\n A. sitting on the table\n B. attached to the wall\n C. attached to an extension cord\n D. attached to a battery pack\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 326, "native_id": "9-550", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 327, "native_id": "8-493", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 327, "native_id": "8-493", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 327, "native_id": "8-493", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\n A. in sticks\n B. in pebbles\n C. in sand\n D. in spiked plants\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 327, "native_id": "8-493", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 328, "native_id": "9-257", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 328, "native_id": "9-257", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 328, "native_id": "9-257", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Why might a polar bear grow white hair?\n A. look fancy\n B. random\n C. blend in\n D. stand out\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 328, "native_id": "9-257", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 329, "native_id": "1239", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 329, "native_id": "1239", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 329, "native_id": "1239", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Xylem\n A. discourages pests from landing on leaves\n B. allows plants to move carbon dioxide from root to stems\n C. carries seedlings from roots to leaves\n D. allows plants to move rain thru their systems\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 329, "native_id": "1239", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 330, "native_id": "869", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 330, "native_id": "869", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 330, "native_id": "869", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A food that is a source of heat is\n A. ramen\n B. salad\n C. ice cream\n D. sushi\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 330, "native_id": "869", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 331, "native_id": "7-1105", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 331, "native_id": "7-1105", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 331, "native_id": "7-1105", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When heat is added to something\n A. contaminates may be destroyed\n B. bacterial can grow more rapidly\n C. viruses may be picked up\n D. the thing loses energy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 331, "native_id": "7-1105", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 332, "native_id": "597", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 332, "native_id": "597", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 332, "native_id": "597", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The heart is an example of\n A. a part of the nervous system\n B. an organ that filters toxins\n C. a self-healing protector from germs\n D. something protected by the skeletal system\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 332, "native_id": "597", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 333, "native_id": "385", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 333, "native_id": "385", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 333, "native_id": "385", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Prey are eaten by\n A. an animal herded by sheep dogs\n B. the animal with a starring role in Bambi\n C. animals known for their memory\n D. the fastest mammal with four legs\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 333, "native_id": "385", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 334, "native_id": "1301", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 334, "native_id": "1301", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 334, "native_id": "1301", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is warm-blooded just like\n A. a snake\n B. a cardinal\n C. a spider\n D. a scorpion\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 334, "native_id": "1301", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 335, "native_id": "9-893", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 335, "native_id": "9-893", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 335, "native_id": "9-893", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A flashlight will need this in order to radiate photons:\n A. radiation\n B. acoustic energy\n C. vibrations\n D. electron flow\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 335, "native_id": "9-893", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 336, "native_id": "9-369", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 336, "native_id": "9-369", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 336, "native_id": "9-369", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: to find out how fast you are going you first need to know\n A. where you're going\n B. distance traveled\n C. distance to travel\n D. home location\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 336, "native_id": "9-369", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 337, "native_id": "9-1026", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 337, "native_id": "9-1026", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 337, "native_id": "9-1026", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\n A. weaken\n B. strengthen\n C. coexist\n D. thrive\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 337, "native_id": "9-1026", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 338, "native_id": "7-424", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 338, "native_id": "7-424", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 338, "native_id": "7-424", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\n A. a dinner jacket\n B. a diving suit\n C. a warm coat\n D. a dress suit\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 338, "native_id": "7-424", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 339, "native_id": "9-259", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 339, "native_id": "9-259", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 339, "native_id": "9-259", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do polar bears survive the cold?\n A. B and D\n B. Double Fur Coats\n C. Cold blooded\n D. Compact ears\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 339, "native_id": "9-259", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 340, "native_id": "9-783", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 340, "native_id": "9-783", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 340, "native_id": "9-783", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A solid is likely to form in extreme\n A. floods\n B. wind\n C. chill\n D. rain\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 340, "native_id": "9-783", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 341, "native_id": "1088", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 341, "native_id": "1088", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 341, "native_id": "1088", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\n A. catching prey\n B. aquatic speed\n C. flying\n D. walking\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 341, "native_id": "1088", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 342, "native_id": "1387", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 342, "native_id": "1387", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 342, "native_id": "1387", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of fire giving off light?\n A. an oven is preheated and the pilot light is lit\n B. a match is lit to light a cigarette\n C. a lit candle in a window signalling to someone\n D. a fire that was put out to send smoke signals\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 342, "native_id": "1387", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 343, "native_id": "7-1062", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 343, "native_id": "7-1062", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 343, "native_id": "7-1062", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The respiratory system works by\n A. directing oxygen from lungs to other organs\n B. pushing air through lungs\n C. moving air in a room\n D. making air quality better\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 343, "native_id": "7-1062", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 344, "native_id": "676", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 344, "native_id": "676", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 344, "native_id": "676", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals have more fat\n A. in the ocean\n B. in human homes\n C. in landfills\n D. in polar areas\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 344, "native_id": "676", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 345, "native_id": "1998", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 345, "native_id": "1998", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 345, "native_id": "1998", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A stick bug uses what to protect itself from predators?\n A. poison\n B. its appearance\n C. speed\n D. hearing\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 345, "native_id": "1998", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 346, "native_id": "1698", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 346, "native_id": "1698", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 346, "native_id": "1698", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Corn is sometimes used to make\n A. a simple alcohol\n B. water\n C. glass\n D. milk\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 346, "native_id": "1698", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 347, "native_id": "490", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 347, "native_id": "490", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 347, "native_id": "490", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\n A. heat energy\n B. light energy\n C. color energy\n D. color transfusion\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 347, "native_id": "490", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 348, "native_id": "844", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 348, "native_id": "844", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 348, "native_id": "844", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Little puppies are a result of:\n A. reproduction ?\n B. pet store sale\n C. a begging child\n D. evolution\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 348, "native_id": "844", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 349, "native_id": "1795", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 349, "native_id": "1795", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 349, "native_id": "1795", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would a Jersey most likely be fed?\n A. hamburger\n B. moles\n C. alfalfa\n D. cow\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 349, "native_id": "1795", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 350, "native_id": "1508", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 350, "native_id": "1508", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 350, "native_id": "1508", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these energy sources generates the least amount of pollution?\n A. coal\n B. windmill\n C. lithium batteries\n D. gasoline\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 350, "native_id": "1508", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 351, "native_id": "9-289", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 351, "native_id": "9-289", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 351, "native_id": "9-289", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Grass snakes live in what?\n A. trees\n B. mountains\n C. lakes\n D. turf\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 351, "native_id": "9-289", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 352, "native_id": "9-668", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 352, "native_id": "9-668", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 352, "native_id": "9-668", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cephalopod ink is by octopuses to\n A. mate\n B. feed\n C. hide\n D. play\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 352, "native_id": "9-668", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 353, "native_id": "7-364", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 353, "native_id": "7-364", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 353, "native_id": "7-364", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Tapping a drumstick to a drum will\n A. reverberate when touched together\n B. vibrate when next to each other\n C. shake around when near\n D. put each other down\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 353, "native_id": "7-364", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 354, "native_id": "1271", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 354, "native_id": "1271", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 354, "native_id": "1271", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Snow is more likely to fall two months before\n A. June\n B. March\n C. September\n D. December\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 354, "native_id": "1271", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 355, "native_id": "9-1117", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 355, "native_id": "9-1117", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 355, "native_id": "9-1117", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\n A. Scorpion\n B. House Fly\n C. Cricket\n D. Moth\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 355, "native_id": "9-1117", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 356, "native_id": "35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 356, "native_id": "35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 356, "native_id": "35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Dead plants are easier to find in\n A. January\n B. July\n C. May\n D. September\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 356, "native_id": "35", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 357, "native_id": "1660", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 357, "native_id": "1660", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 357, "native_id": "1660", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The harder a child pushes a toy car\n A. decreases the distance it will travel\n B. the further it will roll across the floor\n C. the quicker the child will want to play with another toy\n D. determines how long the child with play with it\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 357, "native_id": "1660", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 358, "native_id": "7-710", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 358, "native_id": "7-710", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 358, "native_id": "7-710", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Fossil fuels\n A. come from old age\n B. come from expired life\n C. take two years to create\n D. are created in a year\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 358, "native_id": "7-710", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 359, "native_id": "8-52", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 359, "native_id": "8-52", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 359, "native_id": "8-52", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\n A. a room to have overhead lights\n B. night on Earth to be dimly lit\n C. plastic stars to decorate a ceiling\n D. a person to be the star of a show\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 359, "native_id": "8-52", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 360, "native_id": "9-1167", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 360, "native_id": "9-1167", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 360, "native_id": "9-1167", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Erosion is caused by different kinds of\n A. soil\n B. fish\n C. rocks\n D. weather\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 360, "native_id": "9-1167", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 361, "native_id": "8-43", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 361, "native_id": "8-43", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 361, "native_id": "8-43", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Respiration is a\n A. happens for some species\n B. happens for only land dwelling mammals\n C. occurs for only sea creatures\n D. commonality among all animals\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 361, "native_id": "8-43", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 362, "native_id": "9-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 362, "native_id": "9-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 362, "native_id": "9-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\n A. magical\n B. closed\n C. broken\n D. open\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 362, "native_id": "9-57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 363, "native_id": "1411", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 363, "native_id": "1411", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 363, "native_id": "1411", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears live in\n A. frosty environments\n B. tepid environments\n C. warm environments\n D. tropical environments\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 363, "native_id": "1411", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 364, "native_id": "9-206", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 364, "native_id": "9-206", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 364, "native_id": "9-206", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would be more likely to attract a magnet?\n A. a plastic zipper\n B. flowing water\n C. a car engine\n D. A wooden desk\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 364, "native_id": "9-206", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 365, "native_id": "7-740", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 365, "native_id": "7-740", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 365, "native_id": "7-740", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What constitutes a frog's diet?\n A. it eats all plants\n B. it will eat dogs\n C. it only eats burgers\n D. it chomps on insects\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 365, "native_id": "7-740", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 366, "native_id": "1774", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 366, "native_id": "1774", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 366, "native_id": "1774", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of the digestive system digesting food for the body?\n A. a man eating nachos then getting food poisoning\n B. a baby drinking formula then needing a diaper change\n C. a cat eating food then throwing it up\n D. a horse licking a salt lick\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 366, "native_id": "1774", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 367, "native_id": "7-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 367, "native_id": "7-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 367, "native_id": "7-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The body is negatively impacted by\n A. white blood cells\n B. vitamins\n C. rotavirus\n D. nasal decongestants\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 367, "native_id": "7-93", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 368, "native_id": "8-97", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 368, "native_id": "8-97", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 368, "native_id": "8-97", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\n A. run wire through currants\n B. run a continuous current\n C. run around the wire\n D. currently run wire through\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 368, "native_id": "8-97", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 369, "native_id": "9-813", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 369, "native_id": "9-813", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 369, "native_id": "9-813", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a place has experienced flooding, what could be responsible?\n A. all of these\n B. there has been excess condensed water vapor\n C. the water lacks oxygen\n D. the local deities are angry\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 369, "native_id": "9-813", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 370, "native_id": "9-686", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 370, "native_id": "9-686", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 370, "native_id": "9-686", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What is an example of reproduction?\n A. farming\n B. egg depositing\n C. flying\n D. walking\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 370, "native_id": "9-686", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 371, "native_id": "9-799", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 371, "native_id": "9-799", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 371, "native_id": "9-799", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A place that is snowy has a large amount of\n A. wind\n B. storms\n C. frozen water\n D. rain\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 371, "native_id": "9-799", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 372, "native_id": "1179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 372, "native_id": "1179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 372, "native_id": "1179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ideal abode for crickets is\n A. a small potted plant in a house\n B. a green and lush tree and plant packed area\n C. a briny and warm body of water\n D. a area surrounded by spider webs\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 372, "native_id": "1179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 373, "native_id": "1954", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 373, "native_id": "1954", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 373, "native_id": "1954", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbon steel is always what?\n A. attractive to various objects that contain iron\n B. pleasant with a magnetic personality\n C. made up of iron and pieces of magnets\n D. hard as a magnetizing rod\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 373, "native_id": "1954", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 374, "native_id": "8-403", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 374, "native_id": "8-403", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 374, "native_id": "8-403", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\n A. it is a mammal\n B. calling it a bird is wrong\n C. all of these\n D. it is capable of reproducing\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 374, "native_id": "8-403", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 375, "native_id": "9-576", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 375, "native_id": "9-576", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 375, "native_id": "9-576", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\n A. minerals\n B. mist\n C. clouds\n D. water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 375, "native_id": "9-576", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 376, "native_id": "9-866", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 376, "native_id": "9-866", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 376, "native_id": "9-866", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: as you get closer to something it begins to\n A. shrinks down to nothing\n B. grow in size visually\n C. show a large shadow\n D. rotate in a clockwise direction\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 376, "native_id": "9-866", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 377, "native_id": "7-208", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 377, "native_id": "7-208", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 377, "native_id": "7-208", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: After a storm\n A. ponds may dry out\n B. flowers will wilt and wither\n C. creek beds may be spilling over\n D. drinking water will be in short supply\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 377, "native_id": "7-208", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 378, "native_id": "9-771", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 378, "native_id": "9-771", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 378, "native_id": "9-771", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could I use as biofuel\n A. Gold\n B. Car\n C. Diamonds\n D. Pine Needles\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 378, "native_id": "9-771", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 379, "native_id": "998", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 379, "native_id": "998", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 379, "native_id": "998", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal is hiding from a predator?\n A. a tadpole losing its tail as it grows\n B. an angler fish using its Esca to lure another fish\n C. an octopus mimicking the color and texture of a rocky outcrop\n D. a great white shark breaching the water's surface\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 379, "native_id": "998", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 380, "native_id": "433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 380, "native_id": "433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 380, "native_id": "433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\n A. skating on a rough surface\n B. a full bag swung in circles\n C. a computer powering on\n D. a baker stirring batter\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 380, "native_id": "433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 381, "native_id": "9-508", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 381, "native_id": "9-508", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 381, "native_id": "9-508", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the night sky shows very far away what\n A. clumps of flaming gas\n B. tidal waves washing over beaches\n C. aircraft falling towards collision\n D. party balloons tied to houses\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 381, "native_id": "9-508", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 382, "native_id": "7-561", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 382, "native_id": "7-561", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 382, "native_id": "7-561", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which would you likely find inside a beach ball?\n A. cheese\n B. steam\n C. water\n D. air\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 382, "native_id": "7-561", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 383, "native_id": "7-976", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 383, "native_id": "7-976", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 383, "native_id": "7-976", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Two fridge decorations when touched back to back\n A. shove each other away\n B. are attracted to each other\n C. have very little reaction\n D. are reflective when together\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 383, "native_id": "7-976", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 384, "native_id": "1635", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 384, "native_id": "1635", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 384, "native_id": "1635", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Runoff happens because of\n A. birds\n B. cattails\n C. people\n D. fish\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 384, "native_id": "1635", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 385, "native_id": "7-875", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 385, "native_id": "7-875", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 385, "native_id": "7-875", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Desert environments are generally\n A. sweltering\n B. arctic like\n C. lush\n D. frigid\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 385, "native_id": "7-875", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 386, "native_id": "7-1053", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 386, "native_id": "7-1053", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 386, "native_id": "7-1053", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Are deserts characterized by high sunshine?\n A. they get low sunlight\n B. deserts get surplus sun\n C. deserts get little sun\n D. deserts are always cloudy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 386, "native_id": "7-1053", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 387, "native_id": "9-957", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 387, "native_id": "9-957", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 387, "native_id": "9-957", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water conservation could be a survival tactic in\n A. The Appalachian Mountains\n B. New York City\n C. The Amazon\n D. The Gobi Desert\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 387, "native_id": "9-957", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 388, "native_id": "1150", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 388, "native_id": "1150", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 388, "native_id": "1150", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Objects used to hold sheets of paper together are often\n A. large\n B. wooden\n C. ferromagnetic\n D. electronic\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 388, "native_id": "1150", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 389, "native_id": "8-240", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 389, "native_id": "8-240", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 389, "native_id": "8-240", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\n A. energy\n B. fun\n C. taste\n D. soil\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 389, "native_id": "8-240", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 390, "native_id": "9-554", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 390, "native_id": "9-554", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 390, "native_id": "9-554", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A bear cub learns to stay away from unknown bears because\n A. they are much bigger than the cub\n B. the other bears look like its mother\n C. their mother teaches them to keep their distance\n D. the unknown bears look harmless\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 390, "native_id": "9-554", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 391, "native_id": "9-135", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 391, "native_id": "9-135", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 391, "native_id": "9-135", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: a person driving to work in which of these is most likely to lose control?\n A. a dry cobblestone road\n B. a sleet covered highway\n C. a dry paved road\n D. a dry gravel road\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 391, "native_id": "9-135", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 392, "native_id": "7-1096", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 392, "native_id": "7-1096", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 392, "native_id": "7-1096", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The only creature with offspring that is hatched, of these, is the\n A. squirrel\n B. swallow\n C. mink\n D. bat\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 392, "native_id": "7-1096", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 393, "native_id": "841", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 393, "native_id": "841", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 393, "native_id": "841", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What element is prevalent in a plateau?\n A. helium\n B. krypton\n C. silicon\n D. neon\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 393, "native_id": "841", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 394, "native_id": "7-146", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 394, "native_id": "7-146", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 394, "native_id": "7-146", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When a city tears down a park in a city, the park\n A. is removed\n B. is renewed\n C. is retrieved\n D. is restored\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 394, "native_id": "7-146", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 395, "native_id": "1554", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 395, "native_id": "1554", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 395, "native_id": "1554", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Seeds\n A. are useless shells that need to be discarded\n B. store extra bits of chlorophyll\n C. need to be mashed to grow\n D. aid in feeding what grows from them\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 395, "native_id": "1554", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 396, "native_id": "9-731", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 396, "native_id": "9-731", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 396, "native_id": "9-731", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\n A. consumption\n B. cinematography\n C. direction\n D. production\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 396, "native_id": "9-731", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 397, "native_id": "1780", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 397, "native_id": "1780", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 397, "native_id": "1780", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The spring season brings\n A. Bees\n B. Snow\n C. More Oxygen\n D. Dust\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 397, "native_id": "1780", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 398, "native_id": "7-1077", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 398, "native_id": "7-1077", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 398, "native_id": "7-1077", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Kinetic energy can be found in objects that move, such as\n A. flower pots on a wagon\n B. cars that are in a lot\n C. kids that are sleeping soundly\n D. skateboards that are ridden all day\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 398, "native_id": "7-1077", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 399, "native_id": "8-494", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 399, "native_id": "8-494", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 399, "native_id": "8-494", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these is the quickest to go visiting from our world?\n A. none of these\n B. a trip to mars\n C. a trip to the moon\n D. a trip to the northern star\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 399, "native_id": "8-494", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 400, "native_id": "936", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 400, "native_id": "936", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 400, "native_id": "936", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Animals died after the removal of a\n A. bush\n B. street\n C. house\n D. city\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 400, "native_id": "936", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 401, "native_id": "8-478", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 401, "native_id": "8-478", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 401, "native_id": "8-478", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If I want to go running at night, what can I use as a reflector?\n A. A black shirt\n B. Kitchen foil\n C. Sunglasses\n D. A megaphone\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 401, "native_id": "8-478", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 402, "native_id": "9-669", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 402, "native_id": "9-669", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 402, "native_id": "9-669", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: the closest star to our planet delivers solar energy to the planet\n A. maybe\n B. all of these\n C. this is sure\n D. this is uncertain\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 402, "native_id": "9-669", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 403, "native_id": "7-732", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 403, "native_id": "7-732", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 403, "native_id": "7-732", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\n A. produce energy\n B. use heat energy\n C. burn energy\n D. fuel the world\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 403, "native_id": "7-732", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 404, "native_id": "7-658", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 404, "native_id": "7-658", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 404, "native_id": "7-658", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\n A. wasp\n B. bee\n C. scorpion\n D. butterfly\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 404, "native_id": "7-658", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 405, "native_id": "1003", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 405, "native_id": "1003", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 405, "native_id": "1003", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A renewable resource is\n A. fossil fuel\n B. turbine produced electricity\n C. copper\n D. coal lumps\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 405, "native_id": "1003", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 406, "native_id": "8-62", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 406, "native_id": "8-62", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 406, "native_id": "8-62", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\n A. would cease existing\n B. would be troubled\n C. would be thriving\n D. would be endangered\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 406, "native_id": "8-62", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 407, "native_id": "7-386", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 407, "native_id": "7-386", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 407, "native_id": "7-386", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Acid can be used to make a new\n A. light\n B. substance\n C. electricity\n D. sound\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 407, "native_id": "7-386", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 408, "native_id": "257", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 408, "native_id": "257", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 408, "native_id": "257", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Global warming is lowering the world's amount of\n A. hurricanes\n B. ocean levels\n C. carbon dioxide\n D. ice\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 408, "native_id": "257", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 409, "native_id": "147", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 409, "native_id": "147", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 409, "native_id": "147", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Echolocation can't detect an object's\n A. distance\n B. shape\n C. size\n D. temperature\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 409, "native_id": "147", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 410, "native_id": "7-599", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 410, "native_id": "7-599", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 410, "native_id": "7-599", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What material has already broken down?\n A. wood\n B. glass\n C. boulders\n D. sand\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 410, "native_id": "7-599", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 411, "native_id": "8-92", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 411, "native_id": "8-92", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 411, "native_id": "8-92", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which beverage would dissolve solids the best?\n A. A glass of ice-cold water\n B. A boiling hot mug of tea\n C. A cup of warm milk\n D. A room temperature glass of water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 411, "native_id": "8-92", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 412, "native_id": "354", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 412, "native_id": "354", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 412, "native_id": "354", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which animal has live births?\n A. poodle\n B. hummingbird\n C. crocodile\n D. trout\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 412, "native_id": "354", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 413, "native_id": "9-966", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 413, "native_id": "9-966", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 413, "native_id": "9-966", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Quartz crystals are made up of\n A. majic\n B. hexagons\n C. octogons\n D. water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 413, "native_id": "9-966", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 414, "native_id": "9-612", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 414, "native_id": "9-612", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 414, "native_id": "9-612", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\n A. water\n B. nutrients\n C. mitochondria\n D. gas\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 414, "native_id": "9-612", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 415, "native_id": "9-548", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 415, "native_id": "9-548", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 415, "native_id": "9-548", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Did pasteurization get invented by Thomas Edison?\n A. negative\n B. positive\n C. all of these\n D. maybe it was\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 415, "native_id": "9-548", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 416, "native_id": "9-429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 416, "native_id": "9-429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 416, "native_id": "9-429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A glass of water can undergo a chemical change by adding\n A. a cup of salt\n B. a cup of dirt\n C. a cup of water\n D. a cup of ice\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 416, "native_id": "9-429", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 417, "native_id": "7-95", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 417, "native_id": "7-95", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 417, "native_id": "7-95", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water levels may decrease on cloudless days because\n A. water is warmer than the air\n B. air is warmer than water\n C. moisture is pulled upwards\n D. moisture always tries to rise\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 417, "native_id": "7-95", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 418, "native_id": "1560", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 418, "native_id": "1560", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 418, "native_id": "1560", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: To change an object's shape\n A. rip off a corner portion\n B. lay it flat on a table\n C. color the edges of it\n D. add a piece of tape to it\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 418, "native_id": "1560", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 419, "native_id": "9-461", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 419, "native_id": "9-461", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 419, "native_id": "9-461", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\n A. reports of tornadoes in the area\n B. a dog running across the highway behind Steve's car\n C. a sudden fog moving into the area\n D. ice forming on the road\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 419, "native_id": "9-461", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 420, "native_id": "9-490", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 420, "native_id": "9-490", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 420, "native_id": "9-490", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: DNA is a vehicle for passing\n A. clothes types\n B. school grades\n C. elbow size\n D. language and dialect\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 420, "native_id": "9-490", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 421, "native_id": "9-301", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 421, "native_id": "9-301", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 421, "native_id": "9-301", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\n A. food\n B. sunlight\n C. gas\n D. salt\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 421, "native_id": "9-301", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 422, "native_id": "60", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 422, "native_id": "60", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 422, "native_id": "60", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In general, how many times per month is there a full moon?\n A. twice\n B. three times\n C. once\n D. four times\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 422, "native_id": "60", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 423, "native_id": "9-894", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 423, "native_id": "9-894", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 423, "native_id": "9-894", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What kind of object does light bounce off of?\n A. tadpole\n B. any object\n C. item that reflects\n D. black hole\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 423, "native_id": "9-894", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 424, "native_id": "9-895", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 424, "native_id": "9-895", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 424, "native_id": "9-895", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\n A. the season has been quite dry\n B. There has been a lot of rain\n C. snakes shelter under the brush\n D. People have been walking by the brush on the trails\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 424, "native_id": "9-895", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 425, "native_id": "9-281", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 425, "native_id": "9-281", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 425, "native_id": "9-281", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\n A. the body of the pot is of high temperature\n B. the body of the pot is cold\n C. all of these\n D. the body of the pot is wet\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 425, "native_id": "9-281", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 426, "native_id": "202", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 426, "native_id": "202", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 426, "native_id": "202", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What animal is more difficult for predators to see in water?\n A. a fish\n B. a duck\n C. an octopus\n D. a crab\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 426, "native_id": "202", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 427, "native_id": "1937", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 427, "native_id": "1937", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 427, "native_id": "1937", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A wedge requires\n A. electrical energy\n B. chemical energy\n C. mechanical energy\n D. heat energy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 427, "native_id": "1937", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 428, "native_id": "620", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 428, "native_id": "620", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 428, "native_id": "620", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: In solid phase matter has a/an\n A. concrete configuration\n B. ambiguous form\n C. shapeless form\n D. radioactive glow\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 428, "native_id": "620", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 429, "native_id": "8-142", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 429, "native_id": "8-142", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 429, "native_id": "8-142", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\n A. takes a trip to the desert\n B. climbs a tall mountain\n C. travels to the seaside\n D. visits a remote jungle\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 429, "native_id": "8-142", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 430, "native_id": "7-1138", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 430, "native_id": "7-1138", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 430, "native_id": "7-1138", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\n A. adding salt\n B. adding litter\n C. adding sand\n D. adding water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 430, "native_id": "7-1138", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 431, "native_id": "8-471", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 431, "native_id": "8-471", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 431, "native_id": "8-471", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The appropriate place to put this item is the recycling bin\n A. used motor oil\n B. used soda can\n C. used Styrofoam plates\n D. left over medicine\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 431, "native_id": "8-471", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 432, "native_id": "9-433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 432, "native_id": "9-433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 432, "native_id": "9-433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How many times would someone change the page of a calendar in a year?\n A. 13\n B. 12\n C. 15\n D. 14\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 432, "native_id": "9-433", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 433, "native_id": "1458", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 433, "native_id": "1458", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 433, "native_id": "1458", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A car has the least speed if it\n A. is heavy\n B. is large\n C. is turned off\n D. is small\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 433, "native_id": "1458", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 434, "native_id": "57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 434, "native_id": "57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 434, "native_id": "57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which musical instrument is the same type as a guitar?\n A. flute\n B. cello\n C. drum\n D. trumpet\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 434, "native_id": "57", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 435, "native_id": "605", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 435, "native_id": "605", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 435, "native_id": "605", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The balance result will be number of\n A. kilowatts\n B. kilobytes\n C. kilograms\n D. kilometers\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 435, "native_id": "605", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 436, "native_id": "9-889", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 436, "native_id": "9-889", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 436, "native_id": "9-889", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is a place where a human might live?\n A. igloo\n B. cloud\n C. Mars\n D. the Moon\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 436, "native_id": "9-889", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 437, "native_id": "1890", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 437, "native_id": "1890", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 437, "native_id": "1890", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage is when an organism does what?\n A. reconfigure appearance to blend in\n B. hides its young to avoid prey\n C. changes its shape to appear larger\n D. buries itself to disappear momentarily\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 437, "native_id": "1890", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 438, "native_id": "9-618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 438, "native_id": "9-618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 438, "native_id": "9-618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\n A. xylem\n B. luck\n C. magic\n D. dirt\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 438, "native_id": "9-618", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 439, "native_id": "9-523", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 439, "native_id": "9-523", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 439, "native_id": "9-523", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How does a microscope make small things appear?\n A. humongous\n B. transparent\n C. discolored\n D. distorted\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 439, "native_id": "9-523", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 440, "native_id": "1126", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 440, "native_id": "1126", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 440, "native_id": "1126", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Water can turn to vapor\n A. when a pot of water is placed on an off stove burner\n B. when placing water in a freezer\n C. when boiling eggs on a stove top\n D. when placed in a room temperature setting\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 440, "native_id": "1126", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 441, "native_id": "644", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 441, "native_id": "644", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 441, "native_id": "644", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\n A. white light\n B. conversion\n C. heat\n D. sound\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 441, "native_id": "644", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 442, "native_id": "8-365", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 442, "native_id": "8-365", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 442, "native_id": "8-365", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\n A. seconds\n B. days\n C. weeks\n D. years\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 442, "native_id": "8-365", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 443, "native_id": "9-727", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 443, "native_id": "9-727", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 443, "native_id": "9-727", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Camouflage can be used by animals for hunting\n A. water\n B. trees\n C. air\n D. meals\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 443, "native_id": "9-727", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 444, "native_id": "7-461", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 444, "native_id": "7-461", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 444, "native_id": "7-461", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\n A. broccoli\n B. meat\n C. celery\n D. toast\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 444, "native_id": "7-461", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 445, "native_id": "9-1071", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 445, "native_id": "9-1071", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 445, "native_id": "9-1071", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Hand dryers can also be used to\n A. keep cold drinks cool\n B. dry out clothes after coming in from the rain\n C. hydrate your face and hands\n D. make a damp rag damper\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 445, "native_id": "9-1071", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 446, "native_id": "1918", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 446, "native_id": "1918", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 446, "native_id": "1918", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Polar bears require\n A. a tropical environment\n B. a frigid environment\n C. a tepid environment\n D. a warm environment\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 446, "native_id": "1918", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 447, "native_id": "1038", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 447, "native_id": "1038", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 447, "native_id": "1038", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A measurement of time that is less than a minute is a\n A. day\n B. minute\n C. hour\n D. second\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 447, "native_id": "1038", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 448, "native_id": "9-197", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 448, "native_id": "9-197", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 448, "native_id": "9-197", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What will be more available in an area when rainfall increases?\n A. fire\n B. air\n C. dirt\n D. H2O\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 448, "native_id": "9-197", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 449, "native_id": "1393", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 449, "native_id": "1393", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 449, "native_id": "1393", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When does the first quarter phase of the moon occur?\n A. when you cannot see the moon in the sky at night\n B. after the first phase of the lunar month\n C. after a blue moon\n D. during the full moon\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 449, "native_id": "1393", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 450, "native_id": "7-244", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 450, "native_id": "7-244", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 450, "native_id": "7-244", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An ice cube placed in sunlight will\n A. shrink\n B. change color\n C. grow\n D. freeze\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 450, "native_id": "7-244", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 451, "native_id": "9-916", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 451, "native_id": "9-916", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 451, "native_id": "9-916", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\n A. destroying\n B. conserving\n C. losing\n D. squandering\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 451, "native_id": "9-916", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 452, "native_id": "9-1046", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 452, "native_id": "9-1046", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 452, "native_id": "9-1046", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The skeletal system protects which of these?\n A. liver\n B. eyelashes\n C. finger nails\n D. blood vessels\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 452, "native_id": "9-1046", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 453, "native_id": "167", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 453, "native_id": "167", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 453, "native_id": "167", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What has more gravity force than Earth but less than the sun?\n A. Jupiter\n B. the moon\n C. a space station\n D. a comet\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 453, "native_id": "167", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 454, "native_id": "9-566", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 454, "native_id": "9-566", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 454, "native_id": "9-566", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The dam was put under much more stress after the\n A. party\n B. huge rain storm\n C. drought\n D. breakup.\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 454, "native_id": "9-566", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 455, "native_id": "8-28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 455, "native_id": "8-28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 455, "native_id": "8-28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\n A. CO2, water, and argon\n B. sunlight, oxygen, and fertilizer\n C. CO2, H20, and cloudy skies\n D. CO2, H20, and sun rays\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 455, "native_id": "8-28", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 456, "native_id": "7-179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 456, "native_id": "7-179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 456, "native_id": "7-179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: If a nail is Fe, that nail is\n A. foreign\n B. atomic 26\n C. nickel\n D. atomic 12\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 456, "native_id": "7-179", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 457, "native_id": "389", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 457, "native_id": "389", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 457, "native_id": "389", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: when a circle is torn it is\n A. doubled\n B. changed\n C. a smaller circle\n D. a square\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 457, "native_id": "389", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 458, "native_id": "1528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 458, "native_id": "1528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 458, "native_id": "1528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Wind can cause\n A. leaves to remain on branches\n B. trees to stand perfectly still\n C. dunes at the beach to be depleted\n D. still waters on the ocean\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 458, "native_id": "1528", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 459, "native_id": "1457", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 459, "native_id": "1457", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 459, "native_id": "1457", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What happens as water levels rise?\n A. fish swim more\n B. homes are built\n C. land is taller\n D. beaches shrink\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 459, "native_id": "1457", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 460, "native_id": "1208", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 460, "native_id": "1208", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 460, "native_id": "1208", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An increase in an object's temperature occurs when\n A. an orange is placed in a refrigerator\n B. a steak is removed from the freezer to defrost\n C. a glass of water is moved from counter top to dinner table\n D. an ice tray is placed in a freezer\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 460, "native_id": "1208", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 461, "native_id": "1170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 461, "native_id": "1170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 461, "native_id": "1170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A sousaphone\n A. is ancient\n B. is a frog\n C. makes deep noises\n D. is a smartphone\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 461, "native_id": "1170", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 462, "native_id": "8-409", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 462, "native_id": "8-409", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 462, "native_id": "8-409", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A cooked lobster is\n A. inedible\n B. cold\n C. dead\n D. green\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 462, "native_id": "8-409", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 463, "native_id": "8-307", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 463, "native_id": "8-307", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 463, "native_id": "8-307", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An animal might pant\n A. on a sunny day\n B. during a rain storm\n C. when it is snowing\n D. during the night time\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 463, "native_id": "8-307", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 464, "native_id": "1948", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 464, "native_id": "1948", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 464, "native_id": "1948", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Carnivores\n A. eat foliage and vegetables exclusively\n B. are the bottom of the food chain\n C. require prey to survive\n D. require carbon dioxide to survive\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 464, "native_id": "1948", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 465, "native_id": "661", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 465, "native_id": "661", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 465, "native_id": "661", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The light that appears dimmest is\n A. the light in the hall\n B. a light in the room\n C. a star outside the window\n D. a streetlight outside the window\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 465, "native_id": "661", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 466, "native_id": "7-435", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 466, "native_id": "7-435", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 466, "native_id": "7-435", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant will grow strong if it has\n A. love\n B. heat\n C. earth\n D. sand\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 466, "native_id": "7-435", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 467, "native_id": "8-332", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 467, "native_id": "8-332", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 467, "native_id": "8-332", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: loose soil can be caused by one of these\n A. a koala sitting on a tree\n B. none of these\n C. a worm burrowing through the earth\n D. a bird flying through the air\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 467, "native_id": "8-332", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 468, "native_id": "948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 468, "native_id": "948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 468, "native_id": "948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An instinctual behavior is\n A. dogs rolling over on command\n B. frogs returning to the ponds were they hatched to lay eggs\n C. birds mimicking human speech\n D. seals clapping for treats from trainers\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 468, "native_id": "948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 469, "native_id": "381", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 469, "native_id": "381", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 469, "native_id": "381", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: How do plants reproduce?\n A. seeds\n B. stem\n C. flowers\n D. leaves\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 469, "native_id": "381", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 470, "native_id": "9-759", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 470, "native_id": "9-759", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 470, "native_id": "9-759", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What produce pollen and seeds?\n A. lakes that are frozen over\n B. things you give a loved one in a bouquet\n C. various types of animals\n D. a person that is healthy\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 470, "native_id": "9-759", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 471, "native_id": "8-350", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 471, "native_id": "8-350", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 471, "native_id": "8-350", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: which of these would be most ideal for plant root growth?\n A. a sticky clay soil\n B. soil with worms burrowing around\n C. an arid soil with little looseness\n D. all of these\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 471, "native_id": "8-350", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 472, "native_id": "7-727", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 472, "native_id": "7-727", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 472, "native_id": "7-727", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Having a sense of touch means\n A. I am the water\n B. I am a tree\n C. I am an Ant\n D. I am the Air\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 472, "native_id": "7-727", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 473, "native_id": "850", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 473, "native_id": "850", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 473, "native_id": "850", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Live birth is exemplified in\n A. snakes slithering out of eggs\n B. a calf emerging from a mother giraffe\n C. owlets pecking out of their encasement\n D. sea turtles emerging from their shells\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 473, "native_id": "850", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 474, "native_id": "970", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 474, "native_id": "970", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 474, "native_id": "970", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Cooking peas requires\n A. fresh briny sea water\n B. an unheated stove top\n C. salt and cayenne pepper\n D. turning on a stove top\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 474, "native_id": "970", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 475, "native_id": "7-381", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 475, "native_id": "7-381", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 475, "native_id": "7-381", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Earth revolves around\n A. the moon\n B. outer space\n C. another planet\n D. an energy source\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 475, "native_id": "7-381", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 476, "native_id": "9-436", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 476, "native_id": "9-436", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 476, "native_id": "9-436", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A satellite orbits a\n A. empty space\n B. ocean\n C. terrestrial body\n D. air pocket\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 476, "native_id": "9-436", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 477, "native_id": "9-411", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 477, "native_id": "9-411", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 477, "native_id": "9-411", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\n A. they will increase\n B. nothing will happen\n C. they will shrink\n D. they will double\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 477, "native_id": "9-411", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 478, "native_id": "9-692", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 478, "native_id": "9-692", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 478, "native_id": "9-692", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: There is a heightened threat of landslide in\n A. the desert\n B. The Andes\n C. the ocean\n D. Indiana\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 478, "native_id": "9-692", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 479, "native_id": "1334", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 479, "native_id": "1334", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 479, "native_id": "1334", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: An example of water being an electrical conductor would be what?\n A. lightening hitting water and organisms inside dying\n B. standing in a puddle and avoiding being struck by lightening\n C. standing in a field and getting struck by lightening\n D. grabbing a fence and being shocked\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 479, "native_id": "1334", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 480, "native_id": "9-1160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 480, "native_id": "9-1160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 480, "native_id": "9-1160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What would Occur once between January 1st and December 31st\n A. The moons orbit around the year\n B. One rotation on mercury\n C. The distance between earth and Jupiter when traveling at light speed\n D. A Solar Year on earth\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 480, "native_id": "9-1160", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 481, "native_id": "9-89", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 481, "native_id": "9-89", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 481, "native_id": "9-89", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Burning something that reproduces usually will:\n A. impair its well being in some way\n B. weed out weaker members of the species\n C. speed up its biological functions\n D. increase its population growth\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 481, "native_id": "9-89", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 482, "native_id": "9-1034", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 482, "native_id": "9-1034", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 482, "native_id": "9-1034", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: what does a chipmunk do with acorns\n A. throw them at other chipmunks\n B. leave them where they're found\n C. use them to build shelter\n D. transfer them to the stomach\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 482, "native_id": "9-1034", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 483, "native_id": "8-293", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 483, "native_id": "8-293", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 483, "native_id": "8-293", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\n A. cold air\n B. water vapor\n C. very dry\n D. boiling water\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 483, "native_id": "8-293", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 484, "native_id": "9-652", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 484, "native_id": "9-652", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 484, "native_id": "9-652", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A plant that gets extra minerals such as zinc are probably\n A. planted in zinc pills\n B. plated in the sea\n C. placed in good soil\n D. made out of soil\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 484, "native_id": "9-652", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 485, "native_id": "1391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 485, "native_id": "1391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 485, "native_id": "1391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Which item has a higher altitude?\n A. Tile Floor\n B. Cars\n C. A 6'' Man\n D. A Picture Book\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 485, "native_id": "1391", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 486, "native_id": "9-948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 486, "native_id": "9-948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 486, "native_id": "9-948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\n A. from a live cow\n B. from a live plant\n C. from a volcano cave\n D. from a wild dog\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 486, "native_id": "9-948", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 487, "native_id": "8-213", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 487, "native_id": "8-213", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 487, "native_id": "8-213", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: What could have covered an organism in order to create a trilobite?\n A. Grass\n B. Water\n C. Snow\n D. Sand\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 487, "native_id": "8-213", "label": 3}
+{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 488, "native_id": "162", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 488, "native_id": "162", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 488, "native_id": "162", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A dog is more likely to shiver at\n A. 1 pm\n B. 5 am\n C. 9 am\n D. 6 pm\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 488, "native_id": "162", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 489, "native_id": "1359", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 489, "native_id": "1359", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 489, "native_id": "1359", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\n A. nourishes small protein bits that need to eat with tiny shakes\n B. providing nourishment which enables some growth to vegetation\n C. mixes carbs into soluble plant matter\n D. makes good vegetable protein\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 489, "native_id": "1359", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 490, "native_id": "9-743", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 490, "native_id": "9-743", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 490, "native_id": "9-743", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: where might a bunny live?\n A. a thicket\n B. atop palm trees\n C. a sewer system\n D. a deserted island\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 490, "native_id": "9-743", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 491, "native_id": "9-645", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 491, "native_id": "9-645", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 491, "native_id": "9-645", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\n A. it is a predator\n B. it is a vegetarian\n C. it is a freshwater fish\n D. it is a producer\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 491, "native_id": "9-645", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 492, "native_id": "8-250", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 492, "native_id": "8-250", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 492, "native_id": "8-250", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\n A. oil\n B. deer\n C. bugs\n D. recycled plastic fruit\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 492, "native_id": "8-250", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 493, "native_id": "283", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 493, "native_id": "283", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 493, "native_id": "283", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: The Grand Canyon was formed by\n A. a volcano erupting in 1782\n B. a river named after the 20th state to join the union flowing over time\n C. a river named after the 38th state to join the union flowing over time\n D. the Great Lakes drying up\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 493, "native_id": "283", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 494, "native_id": "8-183", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 494, "native_id": "8-183", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 494, "native_id": "8-183", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\n A. UV rays are harmful\n B. sunlight will be fun\n C. the sun is close\n D. the sun is in space\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 494, "native_id": "8-183", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 495, "native_id": "9-284", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 495, "native_id": "9-284", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 495, "native_id": "9-284", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\n A. scalds\n B. cools\n C. toasts\n D. freezes\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 495, "native_id": "9-284", "label": 0}
+{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 496, "native_id": "7-1186", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 496, "native_id": "7-1186", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 496, "native_id": "7-1186", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Pasta may be cooked in water when\n A. the water is warm\n B. the water is on the stove\n C. water is bubbling from applied warmth\n D. the pasta is very fresh\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 496, "native_id": "7-1186", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 497, "native_id": "926", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 497, "native_id": "926", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 497, "native_id": "926", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: A decrease in diseases\n A. has no impact on a population\n B. leads to more sick people\n C. leads to less sick people\n D. leads to an uptick in emergency room visits\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 497, "native_id": "926", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 498, "native_id": "7-519", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 498, "native_id": "7-519", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 498, "native_id": "7-519", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\n A. insects like big beetles\n B. tiny lifeforms in dirt\n C. small mammals living there\n D. a lot of tiny pebbles\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 498, "native_id": "7-519", "label": 1}
+{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "continuation": " A"}, "idx": 0, "task_name": "openbookqa:mc", "doc_id": 499, "native_id": "7-7", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "continuation": " B"}, "idx": 1, "task_name": "openbookqa:mc", "doc_id": 499, "native_id": "7-7", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "continuation": " C"}, "idx": 2, "task_name": "openbookqa:mc", "doc_id": 499, "native_id": "7-7", "label": 2}
+{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "choices": ["A", "B", "C", "D"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\n A. the mountains seem very close\n B. the mountains are boring\n C. the mountains look the same as from up close\n D. the mountains seem smaller than in photographs\nAnswer: D\n\nQuestion: When food is reduced in the stomach\n A. the mind needs time to digest\n B. take a second to digest what I said\n C. nutrients are being deconstructed\n D. reader's digest is a body of works\nAnswer: C\n\nQuestion: You can make a telescope with a\n A. straw\n B. Glass\n C. Candle\n D. mailing tube\nAnswer: D\n\nQuestion: Poison causes harm to which of the following?\n A. a Tree\n B. a robot\n C. a house\n D. a car\nAnswer: A\n\nQuestion: What happens when mercury is placed in water?\n A. it dissolves\n B. it sinks\n C. it floats\n D. it hardens\nAnswer: B\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\n A. cold\n B. water\n C. heat\n D. humidity\nAnswer:", "continuation": " D"}, "idx": 3, "task_name": "openbookqa:mc", "doc_id": 499, "native_id": "7-7", "label": 2}