{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "continuation": " make more phone calls"}, "idx": 0, "task_name": "openbookqa", "doc_id": 0, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "continuation": " quit eating lunch out"}, "idx": 1, "task_name": "openbookqa", "doc_id": 0, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "continuation": " buy less with monopoly money"}, "idx": 2, "task_name": "openbookqa", "doc_id": 0, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "continuation": " have lunch with friends"}, "idx": 3, "task_name": "openbookqa", "doc_id": 0, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is most likely going to be fog around:\nAnswer:", "continuation": " a marsh"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is most likely going to be fog around:\nAnswer:", "continuation": " a tundra"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is most likely going to be fog around:\nAnswer:", "continuation": " the plains"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is most likely going to be fog around:\nAnswer:", "continuation": " a desert"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Predators eat\nAnswer:", "continuation": " lions"}, "idx": 0, "task_name": "openbookqa", "doc_id": 2, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Predators eat\nAnswer:", "continuation": " humans"}, "idx": 1, "task_name": "openbookqa", "doc_id": 2, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Predators eat\nAnswer:", "continuation": " bunnies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 2, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Predators eat\nAnswer:", "continuation": " grass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 2, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "continuation": " roots may be split"}, "idx": 0, "task_name": "openbookqa", "doc_id": 3, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "continuation": " roots may begin to die"}, "idx": 1, "task_name": "openbookqa", "doc_id": 3, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "continuation": " parts may break the concrete"}, "idx": 2, "task_name": "openbookqa", "doc_id": 3, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "continuation": " roots may fall apart"}, "idx": 3, "task_name": "openbookqa", "doc_id": 3, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An electric car runs on electricity via\nAnswer:", "continuation": " gasoline"}, "idx": 0, "task_name": "openbookqa", "doc_id": 4, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An electric car runs on electricity via\nAnswer:", "continuation": " a power station"}, "idx": 1, "task_name": "openbookqa", "doc_id": 4, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An electric car runs on electricity via\nAnswer:", "continuation": " electrical conductors"}, "idx": 2, "task_name": "openbookqa", "doc_id": 4, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An electric car runs on electricity via\nAnswer:", "continuation": " fuel"}, "idx": 3, "task_name": "openbookqa", "doc_id": 4, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "continuation": " oxygen"}, "idx": 0, "task_name": "openbookqa", "doc_id": 5, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "continuation": " nitrogen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 5, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "continuation": " carbon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 5, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "continuation": " rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 5, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: an electric car contains a motor that runs on\nAnswer:", "continuation": " gas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 6, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: an electric car contains a motor that runs on\nAnswer:", "continuation": " hydrogen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 6, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: an electric car contains a motor that runs on\nAnswer:", "continuation": " ions"}, "idx": 2, "task_name": "openbookqa", "doc_id": 6, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: an electric car contains a motor that runs on\nAnswer:", "continuation": " plutonium"}, "idx": 3, "task_name": "openbookqa", "doc_id": 6, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "continuation": " moons gravity"}, "idx": 0, "task_name": "openbookqa", "doc_id": 7, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "continuation": " human planet rotation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 7, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "continuation": " global warming"}, "idx": 2, "task_name": "openbookqa", "doc_id": 7, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "continuation": " moon rotation"}, "idx": 3, "task_name": "openbookqa", "doc_id": 7, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The summer solstice in the northern hemisphere is four months before\nAnswer:", "continuation": " May"}, "idx": 0, "task_name": "openbookqa", "doc_id": 8, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The summer solstice in the northern hemisphere is four months before\nAnswer:", "continuation": " July"}, "idx": 1, "task_name": "openbookqa", "doc_id": 8, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The summer solstice in the northern hemisphere is four months before\nAnswer:", "continuation": " April"}, "idx": 2, "task_name": "openbookqa", "doc_id": 8, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The summer solstice in the northern hemisphere is four months before\nAnswer:", "continuation": " October"}, "idx": 3, "task_name": "openbookqa", "doc_id": 8, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The main component in dirt is\nAnswer:", "continuation": " microorganisms"}, "idx": 0, "task_name": "openbookqa", "doc_id": 9, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The main component in dirt is\nAnswer:", "continuation": " broken stones"}, "idx": 1, "task_name": "openbookqa", "doc_id": 9, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The main component in dirt is\nAnswer:", "continuation": " pollution"}, "idx": 2, "task_name": "openbookqa", "doc_id": 9, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The main component in dirt is\nAnswer:", "continuation": " bacteria"}, "idx": 3, "task_name": "openbookqa", "doc_id": 9, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It's easier for human's to survive in:\nAnswer:", "continuation": " a cave"}, "idx": 0, "task_name": "openbookqa", "doc_id": 10, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It's easier for human's to survive in:\nAnswer:", "continuation": " the ocean."}, "idx": 1, "task_name": "openbookqa", "doc_id": 10, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It's easier for human's to survive in:\nAnswer:", "continuation": " a town"}, "idx": 2, "task_name": "openbookqa", "doc_id": 10, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It's easier for human's to survive in:\nAnswer:", "continuation": " alone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 10, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cactus stem is used to store\nAnswer:", "continuation": " fruit"}, "idx": 0, "task_name": "openbookqa", "doc_id": 11, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cactus stem is used to store\nAnswer:", "continuation": " liquid"}, "idx": 1, "task_name": "openbookqa", "doc_id": 11, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cactus stem is used to store\nAnswer:", "continuation": " food"}, "idx": 2, "task_name": "openbookqa", "doc_id": 11, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cactus stem is used to store\nAnswer:", "continuation": " spines"}, "idx": 3, "task_name": "openbookqa", "doc_id": 11, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "continuation": " an eagle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 12, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "continuation": " a cow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 12, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "continuation": " a gecko"}, "idx": 2, "task_name": "openbookqa", "doc_id": 12, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "continuation": " a deer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 12, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The chance of wildfires is increased by\nAnswer:", "continuation": " parched foliage"}, "idx": 0, "task_name": "openbookqa", "doc_id": 13, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The chance of wildfires is increased by\nAnswer:", "continuation": " torrential rain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 13, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The chance of wildfires is increased by\nAnswer:", "continuation": " lush foliage"}, "idx": 2, "task_name": "openbookqa", "doc_id": 13, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The chance of wildfires is increased by\nAnswer:", "continuation": " careful fire maintenance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 13, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A positive effect of burning biofuel is\nAnswer:", "continuation": " shortage of crops for the food supply"}, "idx": 0, "task_name": "openbookqa", "doc_id": 14, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A positive effect of burning biofuel is\nAnswer:", "continuation": " an increase in air pollution"}, "idx": 1, "task_name": "openbookqa", "doc_id": 14, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A positive effect of burning biofuel is\nAnswer:", "continuation": " powering the lights in a home"}, "idx": 2, "task_name": "openbookqa", "doc_id": 14, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A positive effect of burning biofuel is\nAnswer:", "continuation": " deforestation in the amazon to make room for crops"}, "idx": 3, "task_name": "openbookqa", "doc_id": 14, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "continuation": " wind power will be expensive"}, "idx": 0, "task_name": "openbookqa", "doc_id": 15, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "continuation": " gas costs will rise"}, "idx": 1, "task_name": "openbookqa", "doc_id": 15, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "continuation": " oil costs will be maintained"}, "idx": 2, "task_name": "openbookqa", "doc_id": 15, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "continuation": " gasoline will be needed less"}, "idx": 3, "task_name": "openbookqa", "doc_id": 15, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "continuation": " sun grafts"}, "idx": 0, "task_name": "openbookqa", "doc_id": 16, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "continuation": " sunlight shields"}, "idx": 1, "task_name": "openbookqa", "doc_id": 16, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "continuation": " panels collecting sunlight"}, "idx": 2, "task_name": "openbookqa", "doc_id": 16, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "continuation": " solar bees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 16, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Mola Mola might live where?\nAnswer:", "continuation": " Lake Michigan"}, "idx": 0, "task_name": "openbookqa", "doc_id": 17, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Mola Mola might live where?\nAnswer:", "continuation": " The Mississippi River"}, "idx": 1, "task_name": "openbookqa", "doc_id": 17, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Mola Mola might live where?\nAnswer:", "continuation": " Bay of Bengal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 17, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Mola Mola might live where?\nAnswer:", "continuation": " Lake Eerie"}, "idx": 3, "task_name": "openbookqa", "doc_id": 17, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which requires energy to move?\nAnswer:", "continuation": " weasel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 18, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which requires energy to move?\nAnswer:", "continuation": " willow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 18, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which requires energy to move?\nAnswer:", "continuation": " mango"}, "idx": 2, "task_name": "openbookqa", "doc_id": 18, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which requires energy to move?\nAnswer:", "continuation": " poison ivy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 18, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal that only eats plants is a\nAnswer:", "continuation": " rat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 19, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal that only eats plants is a\nAnswer:", "continuation": " moth"}, "idx": 1, "task_name": "openbookqa", "doc_id": 19, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal that only eats plants is a\nAnswer:", "continuation": " chimpanzee"}, "idx": 2, "task_name": "openbookqa", "doc_id": 19, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal that only eats plants is a\nAnswer:", "continuation": " pig"}, "idx": 3, "task_name": "openbookqa", "doc_id": 19, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "continuation": " Hanoi"}, "idx": 0, "task_name": "openbookqa", "doc_id": 20, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "continuation": " Athens"}, "idx": 1, "task_name": "openbookqa", "doc_id": 20, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "continuation": " Baghdad"}, "idx": 2, "task_name": "openbookqa", "doc_id": 20, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "continuation": " Phoenix"}, "idx": 3, "task_name": "openbookqa", "doc_id": 20, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of conservation is avoiding the use of\nAnswer:", "continuation": " gasoline"}, "idx": 0, "task_name": "openbookqa", "doc_id": 21, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of conservation is avoiding the use of\nAnswer:", "continuation": " air"}, "idx": 1, "task_name": "openbookqa", "doc_id": 21, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of conservation is avoiding the use of\nAnswer:", "continuation": " snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 21, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of conservation is avoiding the use of\nAnswer:", "continuation": " clothes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 21, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can feathers on Spheniscidae be used for?\nAnswer:", "continuation": " keeping warm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 22, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can feathers on Spheniscidae be used for?\nAnswer:", "continuation": " flying"}, "idx": 1, "task_name": "openbookqa", "doc_id": 22, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can feathers on Spheniscidae be used for?\nAnswer:", "continuation": " sleeping"}, "idx": 2, "task_name": "openbookqa", "doc_id": 22, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can feathers on Spheniscidae be used for?\nAnswer:", "continuation": " eating"}, "idx": 3, "task_name": "openbookqa", "doc_id": 22, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation can cause\nAnswer:", "continuation": " More fresh water for people to drink"}, "idx": 0, "task_name": "openbookqa", "doc_id": 23, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation can cause\nAnswer:", "continuation": " Lower Life Expectancy in Countries"}, "idx": 1, "task_name": "openbookqa", "doc_id": 23, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation can cause\nAnswer:", "continuation": " More food for more people"}, "idx": 2, "task_name": "openbookqa", "doc_id": 23, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation can cause\nAnswer:", "continuation": " More space for places to people to live"}, "idx": 3, "task_name": "openbookqa", "doc_id": 23, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Shining a light through a diamond can\nAnswer:", "continuation": " make a lot of bright lights shine"}, "idx": 0, "task_name": "openbookqa", "doc_id": 24, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Shining a light through a diamond can\nAnswer:", "continuation": " summon a brilliant wave of color"}, "idx": 1, "task_name": "openbookqa", "doc_id": 24, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Shining a light through a diamond can\nAnswer:", "continuation": " heat up a room"}, "idx": 2, "task_name": "openbookqa", "doc_id": 24, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Shining a light through a diamond can\nAnswer:", "continuation": " make a lot of money"}, "idx": 3, "task_name": "openbookqa", "doc_id": 24, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "continuation": " its snout"}, "idx": 0, "task_name": "openbookqa", "doc_id": 25, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "continuation": " its gills"}, "idx": 1, "task_name": "openbookqa", "doc_id": 25, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "continuation": " its nose"}, "idx": 2, "task_name": "openbookqa", "doc_id": 25, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "continuation": " its belly"}, "idx": 3, "task_name": "openbookqa", "doc_id": 25, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would stop a car quicker?\nAnswer:", "continuation": " a wheel with wet brake pads"}, "idx": 0, "task_name": "openbookqa", "doc_id": 26, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would stop a car quicker?\nAnswer:", "continuation": " a wheel without brake pads"}, "idx": 1, "task_name": "openbookqa", "doc_id": 26, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would stop a car quicker?\nAnswer:", "continuation": " a wheel with worn brake pads"}, "idx": 2, "task_name": "openbookqa", "doc_id": 26, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would stop a car quicker?\nAnswer:", "continuation": " a wheel with dry brake pads"}, "idx": 3, "task_name": "openbookqa", "doc_id": 26, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "continuation": " the circulatory system"}, "idx": 0, "task_name": "openbookqa", "doc_id": 27, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "continuation": " the digestive system"}, "idx": 1, "task_name": "openbookqa", "doc_id": 27, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "continuation": " the school system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 27, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "continuation": " central nervous system"}, "idx": 3, "task_name": "openbookqa", "doc_id": 27, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "continuation": " gone"}, "idx": 0, "task_name": "openbookqa", "doc_id": 28, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "continuation": " breaking"}, "idx": 1, "task_name": "openbookqa", "doc_id": 28, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "continuation": " falling"}, "idx": 2, "task_name": "openbookqa", "doc_id": 28, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "continuation": " moving upwards"}, "idx": 3, "task_name": "openbookqa", "doc_id": 28, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When it's flying, a plane has no friction with the\nAnswer:", "continuation": " wings"}, "idx": 0, "task_name": "openbookqa", "doc_id": 29, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When it's flying, a plane has no friction with the\nAnswer:", "continuation": " ground"}, "idx": 1, "task_name": "openbookqa", "doc_id": 29, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When it's flying, a plane has no friction with the\nAnswer:", "continuation": " air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 29, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When it's flying, a plane has no friction with the\nAnswer:", "continuation": " clouds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 29, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To grow plants require\nAnswer:", "continuation": " acid rain"}, "idx": 0, "task_name": "openbookqa", "doc_id": 30, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To grow plants require\nAnswer:", "continuation": " pesticides"}, "idx": 1, "task_name": "openbookqa", "doc_id": 30, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To grow plants require\nAnswer:", "continuation": " shafts of sunlight"}, "idx": 2, "task_name": "openbookqa", "doc_id": 30, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To grow plants require\nAnswer:", "continuation": " moonbeam rays"}, "idx": 3, "task_name": "openbookqa", "doc_id": 30, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the best way to guess a babies eye color?\nAnswer:", "continuation": " The surroundings they are born in."}, "idx": 0, "task_name": "openbookqa", "doc_id": 31, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the best way to guess a babies eye color?\nAnswer:", "continuation": " Their parents usual diet."}, "idx": 1, "task_name": "openbookqa", "doc_id": 31, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the best way to guess a babies eye color?\nAnswer:", "continuation": " Just take a random guess."}, "idx": 2, "task_name": "openbookqa", "doc_id": 31, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the best way to guess a babies eye color?\nAnswer:", "continuation": " The genealogy records of their family."}, "idx": 3, "task_name": "openbookqa", "doc_id": 31, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal eats plants?\nAnswer:", "continuation": " eagles"}, "idx": 0, "task_name": "openbookqa", "doc_id": 32, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal eats plants?\nAnswer:", "continuation": " robins"}, "idx": 1, "task_name": "openbookqa", "doc_id": 32, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal eats plants?\nAnswer:", "continuation": " owls"}, "idx": 2, "task_name": "openbookqa", "doc_id": 32, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal eats plants?\nAnswer:", "continuation": " leopards"}, "idx": 3, "task_name": "openbookqa", "doc_id": 32, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a hypothesis?\nAnswer:", "continuation": " The ice caps will completely melt if global warming continues"}, "idx": 0, "task_name": "openbookqa", "doc_id": 33, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a hypothesis?\nAnswer:", "continuation": " The earth is round"}, "idx": 1, "task_name": "openbookqa", "doc_id": 33, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a hypothesis?\nAnswer:", "continuation": " The earth revolves around the sun"}, "idx": 2, "task_name": "openbookqa", "doc_id": 33, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a hypothesis?\nAnswer:", "continuation": " Gravity causes objects to fall"}, "idx": 3, "task_name": "openbookqa", "doc_id": 33, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What explains the characteristic lunar formations?\nAnswer:", "continuation": " remains of ancient ponds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 34, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What explains the characteristic lunar formations?\nAnswer:", "continuation": " many collisions that have occured"}, "idx": 1, "task_name": "openbookqa", "doc_id": 34, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What explains the characteristic lunar formations?\nAnswer:", "continuation": " volcanic explosions over millions of years"}, "idx": 2, "task_name": "openbookqa", "doc_id": 34, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What explains the characteristic lunar formations?\nAnswer:", "continuation": " sink holes due to the moons porous nature"}, "idx": 3, "task_name": "openbookqa", "doc_id": 34, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tadpoles start their lives as\nAnswer:", "continuation": " Water animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 35, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tadpoles start their lives as\nAnswer:", "continuation": " Frogs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 35, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tadpoles start their lives as\nAnswer:", "continuation": " Ants"}, "idx": 2, "task_name": "openbookqa", "doc_id": 35, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tadpoles start their lives as\nAnswer:", "continuation": " College Students"}, "idx": 3, "task_name": "openbookqa", "doc_id": 35, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "continuation": " the apple sitting on a sunny sidewalk"}, "idx": 0, "task_name": "openbookqa", "doc_id": 36, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "continuation": " the apple in the freezer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 36, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "continuation": " the apple sitting on the shaded stoop"}, "idx": 2, "task_name": "openbookqa", "doc_id": 36, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "continuation": " the apple in a closet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 36, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is used for sensing visual things?\nAnswer:", "continuation": " nerves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 37, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is used for sensing visual things?\nAnswer:", "continuation": " tibia"}, "idx": 1, "task_name": "openbookqa", "doc_id": 37, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is used for sensing visual things?\nAnswer:", "continuation": " nostril"}, "idx": 2, "task_name": "openbookqa", "doc_id": 37, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is used for sensing visual things?\nAnswer:", "continuation": " cornea"}, "idx": 3, "task_name": "openbookqa", "doc_id": 37, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They studied the soil by using\nAnswer:", "continuation": " plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 38, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They studied the soil by using\nAnswer:", "continuation": " a telescope"}, "idx": 1, "task_name": "openbookqa", "doc_id": 38, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They studied the soil by using\nAnswer:", "continuation": " roots"}, "idx": 2, "task_name": "openbookqa", "doc_id": 38, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They studied the soil by using\nAnswer:", "continuation": " a microscope"}, "idx": 3, "task_name": "openbookqa", "doc_id": 38, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill's arm got cold when he put it inside the\nAnswer:", "continuation": " refrigerator"}, "idx": 0, "task_name": "openbookqa", "doc_id": 39, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill's arm got cold when he put it inside the\nAnswer:", "continuation": " room"}, "idx": 1, "task_name": "openbookqa", "doc_id": 39, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill's arm got cold when he put it inside the\nAnswer:", "continuation": " jacket"}, "idx": 2, "task_name": "openbookqa", "doc_id": 39, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill's arm got cold when he put it inside the\nAnswer:", "continuation": " oven"}, "idx": 3, "task_name": "openbookqa", "doc_id": 39, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A recyclable material can be\nAnswer:", "continuation": " transformed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 40, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A recyclable material can be\nAnswer:", "continuation": " traded"}, "idx": 1, "task_name": "openbookqa", "doc_id": 40, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A recyclable material can be\nAnswer:", "continuation": " thrown away"}, "idx": 2, "task_name": "openbookqa", "doc_id": 40, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A recyclable material can be\nAnswer:", "continuation": " used more times"}, "idx": 3, "task_name": "openbookqa", "doc_id": 40, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is different about birth in humans and chickens?\nAnswer:", "continuation": " Mother"}, "idx": 0, "task_name": "openbookqa", "doc_id": 41, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is different about birth in humans and chickens?\nAnswer:", "continuation": " Fertilization"}, "idx": 1, "task_name": "openbookqa", "doc_id": 41, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is different about birth in humans and chickens?\nAnswer:", "continuation": " Father"}, "idx": 2, "task_name": "openbookqa", "doc_id": 41, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is different about birth in humans and chickens?\nAnswer:", "continuation": " the hard shell"}, "idx": 3, "task_name": "openbookqa", "doc_id": 41, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these situations is an example of pollutants?\nAnswer:", "continuation": " plastic bags floating in the ocean"}, "idx": 0, "task_name": "openbookqa", "doc_id": 42, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these situations is an example of pollutants?\nAnswer:", "continuation": " mallard ducks floating on a lake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 42, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these situations is an example of pollutants?\nAnswer:", "continuation": " cottonwood seeds floating in the air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 42, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these situations is an example of pollutants?\nAnswer:", "continuation": " cirrus clouds floating in the sky"}, "idx": 3, "task_name": "openbookqa", "doc_id": 42, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Human reproduction requires\nAnswer:", "continuation": " eggs with shells"}, "idx": 0, "task_name": "openbookqa", "doc_id": 43, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Human reproduction requires\nAnswer:", "continuation": " nest incubation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 43, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Human reproduction requires\nAnswer:", "continuation": " a nest"}, "idx": 2, "task_name": "openbookqa", "doc_id": 43, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Human reproduction requires\nAnswer:", "continuation": " a womb"}, "idx": 3, "task_name": "openbookqa", "doc_id": 43, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Thermometers\nAnswer:", "continuation": " can help you monitor a fever"}, "idx": 0, "task_name": "openbookqa", "doc_id": 44, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Thermometers\nAnswer:", "continuation": " indicate levels of mercury in the blood"}, "idx": 1, "task_name": "openbookqa", "doc_id": 44, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Thermometers\nAnswer:", "continuation": " read exactly at 98.6 degrees"}, "idx": 2, "task_name": "openbookqa", "doc_id": 44, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Thermometers\nAnswer:", "continuation": " are used only for babies"}, "idx": 3, "task_name": "openbookqa", "doc_id": 44, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "continuation": " someone would turn up the room heater"}, "idx": 0, "task_name": "openbookqa", "doc_id": 45, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "continuation": " someone would turn up the air conditioner"}, "idx": 1, "task_name": "openbookqa", "doc_id": 45, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "continuation": " someone would turn up the music"}, "idx": 2, "task_name": "openbookqa", "doc_id": 45, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "continuation": " someone would turn on the light"}, "idx": 3, "task_name": "openbookqa", "doc_id": 45, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would happen when balloons heat up?\nAnswer:", "continuation": " they get bigger"}, "idx": 0, "task_name": "openbookqa", "doc_id": 46, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would happen when balloons heat up?\nAnswer:", "continuation": " they get smaller"}, "idx": 1, "task_name": "openbookqa", "doc_id": 46, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would happen when balloons heat up?\nAnswer:", "continuation": " nothing happens"}, "idx": 2, "task_name": "openbookqa", "doc_id": 46, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would happen when balloons heat up?\nAnswer:", "continuation": " they fall down"}, "idx": 3, "task_name": "openbookqa", "doc_id": 46, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "continuation": " expand"}, "idx": 0, "task_name": "openbookqa", "doc_id": 47, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "continuation": " melt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 47, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "continuation": " shrink"}, "idx": 2, "task_name": "openbookqa", "doc_id": 47, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "continuation": " fall"}, "idx": 3, "task_name": "openbookqa", "doc_id": 47, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seals are most likely to be found in what type of environment?\nAnswer:", "continuation": " desert"}, "idx": 0, "task_name": "openbookqa", "doc_id": 48, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seals are most likely to be found in what type of environment?\nAnswer:", "continuation": " arctic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 48, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seals are most likely to be found in what type of environment?\nAnswer:", "continuation": " Mediterranean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 48, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seals are most likely to be found in what type of environment?\nAnswer:", "continuation": " tropical"}, "idx": 3, "task_name": "openbookqa", "doc_id": 48, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When the eggs hatch, the offspring are\nAnswer:", "continuation": " killed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 49, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When the eggs hatch, the offspring are\nAnswer:", "continuation": " hurt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 49, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When the eggs hatch, the offspring are\nAnswer:", "continuation": " born"}, "idx": 2, "task_name": "openbookqa", "doc_id": 49, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When the eggs hatch, the offspring are\nAnswer:", "continuation": " cold"}, "idx": 3, "task_name": "openbookqa", "doc_id": 49, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some berries may be eaten by\nAnswer:", "continuation": " a bear or person"}, "idx": 0, "task_name": "openbookqa", "doc_id": 50, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some berries may be eaten by\nAnswer:", "continuation": " a bear or shark"}, "idx": 1, "task_name": "openbookqa", "doc_id": 50, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some berries may be eaten by\nAnswer:", "continuation": " a bear or lion"}, "idx": 2, "task_name": "openbookqa", "doc_id": 50, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some berries may be eaten by\nAnswer:", "continuation": " a bear or wolf"}, "idx": 3, "task_name": "openbookqa", "doc_id": 50, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person has a chance to experience an equinox\nAnswer:", "continuation": " weekly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 51, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person has a chance to experience an equinox\nAnswer:", "continuation": " monthly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 51, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person has a chance to experience an equinox\nAnswer:", "continuation": " annually"}, "idx": 2, "task_name": "openbookqa", "doc_id": 51, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person has a chance to experience an equinox\nAnswer:", "continuation": " biannually"}, "idx": 3, "task_name": "openbookqa", "doc_id": 51, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation of an organism can\nAnswer:", "continuation": " strain the resources of an ecosystem"}, "idx": 0, "task_name": "openbookqa", "doc_id": 52, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation of an organism can\nAnswer:", "continuation": " cause boundless growth of resources"}, "idx": 1, "task_name": "openbookqa", "doc_id": 52, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation of an organism can\nAnswer:", "continuation": " lead to extinction of the organism"}, "idx": 2, "task_name": "openbookqa", "doc_id": 52, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Overpopulation of an organism can\nAnswer:", "continuation": " cause the ecosystem to flourish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 52, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To improve health, what is a good strategy?\nAnswer:", "continuation": " high risk lifestyle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 53, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To improve health, what is a good strategy?\nAnswer:", "continuation": " restaurant food"}, "idx": 1, "task_name": "openbookqa", "doc_id": 53, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To improve health, what is a good strategy?\nAnswer:", "continuation": " business trip"}, "idx": 2, "task_name": "openbookqa", "doc_id": 53, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To improve health, what is a good strategy?\nAnswer:", "continuation": " a spa trip"}, "idx": 3, "task_name": "openbookqa", "doc_id": 53, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A girl and her mom have the same\nAnswer:", "continuation": " date of birth"}, "idx": 0, "task_name": "openbookqa", "doc_id": 54, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A girl and her mom have the same\nAnswer:", "continuation": " shirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 54, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A girl and her mom have the same\nAnswer:", "continuation": " number of toenails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 54, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A girl and her mom have the same\nAnswer:", "continuation": " hair length"}, "idx": 3, "task_name": "openbookqa", "doc_id": 54, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The transportation with the most mass is likely a\nAnswer:", "continuation": " commercial plane"}, "idx": 0, "task_name": "openbookqa", "doc_id": 55, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The transportation with the most mass is likely a\nAnswer:", "continuation": " private plane"}, "idx": 1, "task_name": "openbookqa", "doc_id": 55, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The transportation with the most mass is likely a\nAnswer:", "continuation": " bus"}, "idx": 2, "task_name": "openbookqa", "doc_id": 55, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The transportation with the most mass is likely a\nAnswer:", "continuation": " private car"}, "idx": 3, "task_name": "openbookqa", "doc_id": 55, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit may enjoy\nAnswer:", "continuation": " meat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 56, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit may enjoy\nAnswer:", "continuation": " compost"}, "idx": 1, "task_name": "openbookqa", "doc_id": 56, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit may enjoy\nAnswer:", "continuation": " peas"}, "idx": 2, "task_name": "openbookqa", "doc_id": 56, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit may enjoy\nAnswer:", "continuation": " pebbles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 56, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "continuation": " the deer are moved to a zoo"}, "idx": 0, "task_name": "openbookqa", "doc_id": 57, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "continuation": " the trail is expanded"}, "idx": 1, "task_name": "openbookqa", "doc_id": 57, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "continuation": " the mall has a nature park in it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 57, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "continuation": " the wildlife environment is destroyed"}, "idx": 3, "task_name": "openbookqa", "doc_id": 57, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Owls are likely to hunt at\nAnswer:", "continuation": " 3pm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 58, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Owls are likely to hunt at\nAnswer:", "continuation": " 2am"}, "idx": 1, "task_name": "openbookqa", "doc_id": 58, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Owls are likely to hunt at\nAnswer:", "continuation": " 6pm"}, "idx": 2, "task_name": "openbookqa", "doc_id": 58, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Owls are likely to hunt at\nAnswer:", "continuation": " 7am"}, "idx": 3, "task_name": "openbookqa", "doc_id": 58, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could be a positive aspect of a tree being cut down?\nAnswer:", "continuation": " the plants that were under the tree will have access to more light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 59, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could be a positive aspect of a tree being cut down?\nAnswer:", "continuation": " the squirrels that were in that tree will have an easier time getting to their home"}, "idx": 1, "task_name": "openbookqa", "doc_id": 59, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could be a positive aspect of a tree being cut down?\nAnswer:", "continuation": " Plants under the tree will get cooled off by the shade"}, "idx": 2, "task_name": "openbookqa", "doc_id": 59, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could be a positive aspect of a tree being cut down?\nAnswer:", "continuation": " The sun will shine brighter than before"}, "idx": 3, "task_name": "openbookqa", "doc_id": 59, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Birds carrying away fruit helps the tree\nAnswer:", "continuation": " grow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 60, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Birds carrying away fruit helps the tree\nAnswer:", "continuation": " fertilize"}, "idx": 1, "task_name": "openbookqa", "doc_id": 60, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Birds carrying away fruit helps the tree\nAnswer:", "continuation": " reproduce"}, "idx": 2, "task_name": "openbookqa", "doc_id": 60, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Birds carrying away fruit helps the tree\nAnswer:", "continuation": " conquer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 60, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "continuation": " the UFO is calling"}, "idx": 0, "task_name": "openbookqa", "doc_id": 61, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "continuation": " the UFO had been close"}, "idx": 1, "task_name": "openbookqa", "doc_id": 61, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "continuation": " the UFO is approaching"}, "idx": 2, "task_name": "openbookqa", "doc_id": 61, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "continuation": " the UFO is leaving"}, "idx": 3, "task_name": "openbookqa", "doc_id": 61, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited behavior is exhibited when\nAnswer:", "continuation": " bears take a long winter sleep"}, "idx": 0, "task_name": "openbookqa", "doc_id": 62, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited behavior is exhibited when\nAnswer:", "continuation": " dogs sit on command"}, "idx": 1, "task_name": "openbookqa", "doc_id": 62, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited behavior is exhibited when\nAnswer:", "continuation": " seals clap for their trainers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 62, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited behavior is exhibited when\nAnswer:", "continuation": " rats navigate thru a maze"}, "idx": 3, "task_name": "openbookqa", "doc_id": 62, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What likely explains deforestation?\nAnswer:", "continuation": " Increased insect populations"}, "idx": 0, "task_name": "openbookqa", "doc_id": 63, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What likely explains deforestation?\nAnswer:", "continuation": " Clearing for farming"}, "idx": 1, "task_name": "openbookqa", "doc_id": 63, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What likely explains deforestation?\nAnswer:", "continuation": " reduction in rainfall"}, "idx": 2, "task_name": "openbookqa", "doc_id": 63, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What likely explains deforestation?\nAnswer:", "continuation": " More carbon dioxide"}, "idx": 3, "task_name": "openbookqa", "doc_id": 63, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "continuation": " steak"}, "idx": 0, "task_name": "openbookqa", "doc_id": 64, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "continuation": " blood"}, "idx": 1, "task_name": "openbookqa", "doc_id": 64, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "continuation": " nice weather"}, "idx": 2, "task_name": "openbookqa", "doc_id": 64, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "continuation": " taking food"}, "idx": 3, "task_name": "openbookqa", "doc_id": 64, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The surface of the moon contains\nAnswer:", "continuation": " dogs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 65, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The surface of the moon contains\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 65, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The surface of the moon contains\nAnswer:", "continuation": " high peaks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 65, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The surface of the moon contains\nAnswer:", "continuation": " humans"}, "idx": 3, "task_name": "openbookqa", "doc_id": 65, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "continuation": " four boxes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 66, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "continuation": " eight boxes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 66, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "continuation": " two boxes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 66, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "continuation": " six boxes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 66, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "continuation": " asks for milk"}, "idx": 0, "task_name": "openbookqa", "doc_id": 67, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "continuation": " asks for television"}, "idx": 1, "task_name": "openbookqa", "doc_id": 67, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "continuation": " asks for water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 67, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "continuation": " asks for sleep"}, "idx": 3, "task_name": "openbookqa", "doc_id": 67, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "continuation": " roots"}, "idx": 0, "task_name": "openbookqa", "doc_id": 68, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "continuation": " tensile strength"}, "idx": 1, "task_name": "openbookqa", "doc_id": 68, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "continuation": " plant temperature"}, "idx": 2, "task_name": "openbookqa", "doc_id": 68, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "continuation": " plant color"}, "idx": 3, "task_name": "openbookqa", "doc_id": 68, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "continuation": " rainy sky to fly in"}, "idx": 0, "task_name": "openbookqa", "doc_id": 69, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "continuation": " fast truck to drive"}, "idx": 1, "task_name": "openbookqa", "doc_id": 69, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "continuation": " a car with gasoline"}, "idx": 2, "task_name": "openbookqa", "doc_id": 69, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "continuation": " surfaces to reflect sound off"}, "idx": 3, "task_name": "openbookqa", "doc_id": 69, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "continuation": " puppies have soft fur"}, "idx": 0, "task_name": "openbookqa", "doc_id": 70, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "continuation": " kittens look like their parents"}, "idx": 1, "task_name": "openbookqa", "doc_id": 70, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "continuation": " all mammals are born with fur"}, "idx": 2, "task_name": "openbookqa", "doc_id": 70, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "continuation": " baby rats are mostly bald"}, "idx": 3, "task_name": "openbookqa", "doc_id": 70, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Decaying vegetation is part of the process that\nAnswer:", "continuation": " enables nuclear power to function"}, "idx": 0, "task_name": "openbookqa", "doc_id": 71, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Decaying vegetation is part of the process that\nAnswer:", "continuation": " enables to emitting of light beams"}, "idx": 1, "task_name": "openbookqa", "doc_id": 71, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Decaying vegetation is part of the process that\nAnswer:", "continuation": " enables gas powered motors to operate"}, "idx": 2, "task_name": "openbookqa", "doc_id": 71, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Decaying vegetation is part of the process that\nAnswer:", "continuation": " enables windmills to power electric grids"}, "idx": 3, "task_name": "openbookqa", "doc_id": 71, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "continuation": " melted"}, "idx": 0, "task_name": "openbookqa", "doc_id": 72, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "continuation": " dehydrated"}, "idx": 1, "task_name": "openbookqa", "doc_id": 72, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "continuation": " bloated"}, "idx": 2, "task_name": "openbookqa", "doc_id": 72, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "continuation": " salted"}, "idx": 3, "task_name": "openbookqa", "doc_id": 72, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "continuation": " inverse"}, "idx": 0, "task_name": "openbookqa", "doc_id": 73, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "continuation": " reverse"}, "idx": 1, "task_name": "openbookqa", "doc_id": 73, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "continuation": " direct"}, "idx": 2, "task_name": "openbookqa", "doc_id": 73, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "continuation": " equal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 73, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A fallen leaf\nAnswer:", "continuation": " will turn into a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 74, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A fallen leaf\nAnswer:", "continuation": " will become bright green"}, "idx": 1, "task_name": "openbookqa", "doc_id": 74, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A fallen leaf\nAnswer:", "continuation": " will begin to recycle the nutrients that made up its structure"}, "idx": 2, "task_name": "openbookqa", "doc_id": 74, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A fallen leaf\nAnswer:", "continuation": " is likely to continue to grow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 74, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Over a period of time the weather can change\nAnswer:", "continuation": " The color of my hair"}, "idx": 0, "task_name": "openbookqa", "doc_id": 75, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Over a period of time the weather can change\nAnswer:", "continuation": " The way I walk"}, "idx": 1, "task_name": "openbookqa", "doc_id": 75, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Over a period of time the weather can change\nAnswer:", "continuation": " The size of a statue"}, "idx": 2, "task_name": "openbookqa", "doc_id": 75, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Over a period of time the weather can change\nAnswer:", "continuation": " The sound a computer makes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 75, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plant growth may cause\nAnswer:", "continuation": " an uptick in the number of leaves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 76, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plant growth may cause\nAnswer:", "continuation": " a surge in leaf disease"}, "idx": 1, "task_name": "openbookqa", "doc_id": 76, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plant growth may cause\nAnswer:", "continuation": " a gradual decrease in leaves"}, "idx": 2, "task_name": "openbookqa", "doc_id": 76, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plant growth may cause\nAnswer:", "continuation": " a rapid decline of the leaves"}, "idx": 3, "task_name": "openbookqa", "doc_id": 76, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "continuation": " ice cube"}, "idx": 0, "task_name": "openbookqa", "doc_id": 77, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 77, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "continuation": " electrical transformer"}, "idx": 2, "task_name": "openbookqa", "doc_id": 77, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "continuation": " grass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 77, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which substance is capable of dripping?\nAnswer:", "continuation": " Oxygen"}, "idx": 0, "task_name": "openbookqa", "doc_id": 78, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which substance is capable of dripping?\nAnswer:", "continuation": " Juice"}, "idx": 1, "task_name": "openbookqa", "doc_id": 78, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which substance is capable of dripping?\nAnswer:", "continuation": " Wood"}, "idx": 2, "task_name": "openbookqa", "doc_id": 78, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which substance is capable of dripping?\nAnswer:", "continuation": " Lightning"}, "idx": 3, "task_name": "openbookqa", "doc_id": 78, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If bacon is left too long on a hot stove top\nAnswer:", "continuation": " it will be cooked perfectly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 79, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If bacon is left too long on a hot stove top\nAnswer:", "continuation": " it will be bacteria laden"}, "idx": 1, "task_name": "openbookqa", "doc_id": 79, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If bacon is left too long on a hot stove top\nAnswer:", "continuation": " it will become blackened"}, "idx": 2, "task_name": "openbookqa", "doc_id": 79, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If bacon is left too long on a hot stove top\nAnswer:", "continuation": " it will be left raw"}, "idx": 3, "task_name": "openbookqa", "doc_id": 79, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "continuation": " set to calories"}, "idx": 0, "task_name": "openbookqa", "doc_id": 80, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "continuation": " set to volume"}, "idx": 1, "task_name": "openbookqa", "doc_id": 80, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "continuation": " set to kilometers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 80, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "continuation": " set to width"}, "idx": 3, "task_name": "openbookqa", "doc_id": 80, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: are explosions safe?\nAnswer:", "continuation": " they could harm living things"}, "idx": 0, "task_name": "openbookqa", "doc_id": 81, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: are explosions safe?\nAnswer:", "continuation": " they are very safe"}, "idx": 1, "task_name": "openbookqa", "doc_id": 81, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: are explosions safe?\nAnswer:", "continuation": " they cause nothing serious"}, "idx": 2, "task_name": "openbookqa", "doc_id": 81, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: are explosions safe?\nAnswer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 81, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lowest temperature on the trip was at\nAnswer:", "continuation": " the mountain pass"}, "idx": 0, "task_name": "openbookqa", "doc_id": 82, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lowest temperature on the trip was at\nAnswer:", "continuation": " the plain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 82, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lowest temperature on the trip was at\nAnswer:", "continuation": " the large hill"}, "idx": 2, "task_name": "openbookqa", "doc_id": 82, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lowest temperature on the trip was at\nAnswer:", "continuation": " the canyon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 82, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "continuation": " they are easier to make"}, "idx": 0, "task_name": "openbookqa", "doc_id": 83, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "continuation": " they let in less light"}, "idx": 1, "task_name": "openbookqa", "doc_id": 83, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "continuation": " they are cheaper to produce"}, "idx": 2, "task_name": "openbookqa", "doc_id": 83, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "continuation": " they are much stronger"}, "idx": 3, "task_name": "openbookqa", "doc_id": 83, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Ocean water contains\nAnswer:", "continuation": " copious amounts of seltzer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 84, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Ocean water contains\nAnswer:", "continuation": " scant amounts of sodium chloride"}, "idx": 1, "task_name": "openbookqa", "doc_id": 84, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Ocean water contains\nAnswer:", "continuation": " scant amounts of carbonation"}, "idx": 2, "task_name": "openbookqa", "doc_id": 84, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Ocean water contains\nAnswer:", "continuation": " copious amounts of the combination of Na and Cl"}, "idx": 3, "task_name": "openbookqa", "doc_id": 84, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "continuation": " blood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 85, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 85, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "continuation": " prey"}, "idx": 2, "task_name": "openbookqa", "doc_id": 85, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "continuation": " spots"}, "idx": 3, "task_name": "openbookqa", "doc_id": 85, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Beak shape can influence a bird's ability\nAnswer:", "continuation": " to give birth to live young"}, "idx": 0, "task_name": "openbookqa", "doc_id": 86, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Beak shape can influence a bird's ability\nAnswer:", "continuation": " to mate with it's partner"}, "idx": 1, "task_name": "openbookqa", "doc_id": 86, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Beak shape can influence a bird's ability\nAnswer:", "continuation": " to fly to warmer climates"}, "idx": 2, "task_name": "openbookqa", "doc_id": 86, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Beak shape can influence a bird's ability\nAnswer:", "continuation": " to chew up certain worms"}, "idx": 3, "task_name": "openbookqa", "doc_id": 86, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some blind people have demonstrated bat-like skills by:\nAnswer:", "continuation": " sensing shapes by light and shadows"}, "idx": 0, "task_name": "openbookqa", "doc_id": 87, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some blind people have demonstrated bat-like skills by:\nAnswer:", "continuation": " having a unusually strong sense of smell"}, "idx": 1, "task_name": "openbookqa", "doc_id": 87, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some blind people have demonstrated bat-like skills by:\nAnswer:", "continuation": " sensing nearby objects by temperature change"}, "idx": 2, "task_name": "openbookqa", "doc_id": 87, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some blind people have demonstrated bat-like skills by:\nAnswer:", "continuation": " using sound to 'see'"}, "idx": 3, "task_name": "openbookqa", "doc_id": 87, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "continuation": " A session"}, "idx": 0, "task_name": "openbookqa", "doc_id": 88, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "continuation": " A Match"}, "idx": 1, "task_name": "openbookqa", "doc_id": 88, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "continuation": " A Season"}, "idx": 2, "task_name": "openbookqa", "doc_id": 88, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "continuation": " Autumn"}, "idx": 3, "task_name": "openbookqa", "doc_id": 88, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds provide new plants with\nAnswer:", "continuation": " life sustaining elements"}, "idx": 0, "task_name": "openbookqa", "doc_id": 89, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds provide new plants with\nAnswer:", "continuation": " essentials for photosynthesis"}, "idx": 1, "task_name": "openbookqa", "doc_id": 89, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds provide new plants with\nAnswer:", "continuation": " water and hydrogen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 89, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds provide new plants with\nAnswer:", "continuation": " storage for roots"}, "idx": 3, "task_name": "openbookqa", "doc_id": 89, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "continuation": " ecru"}, "idx": 0, "task_name": "openbookqa", "doc_id": 90, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "continuation": " red"}, "idx": 1, "task_name": "openbookqa", "doc_id": 90, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "continuation": " black"}, "idx": 2, "task_name": "openbookqa", "doc_id": 90, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "continuation": " navy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 90, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The salamander could eat a large amounts of what?\nAnswer:", "continuation": " fettuccine if left around"}, "idx": 0, "task_name": "openbookqa", "doc_id": 91, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The salamander could eat a large amounts of what?\nAnswer:", "continuation": " waxy leaves from certain plants"}, "idx": 1, "task_name": "openbookqa", "doc_id": 91, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The salamander could eat a large amounts of what?\nAnswer:", "continuation": " dead carcass meat from livestock"}, "idx": 2, "task_name": "openbookqa", "doc_id": 91, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The salamander could eat a large amounts of what?\nAnswer:", "continuation": " six legged winged organisms"}, "idx": 3, "task_name": "openbookqa", "doc_id": 91, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person can see\nAnswer:", "continuation": " a radio recording"}, "idx": 0, "task_name": "openbookqa", "doc_id": 92, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person can see\nAnswer:", "continuation": " an emotion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 92, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person can see\nAnswer:", "continuation": " a written message"}, "idx": 2, "task_name": "openbookqa", "doc_id": 92, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person can see\nAnswer:", "continuation": " an abstract idea"}, "idx": 3, "task_name": "openbookqa", "doc_id": 92, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When looking for good soil for plants, typically what is optimal?\nAnswer:", "continuation": " malleable and nutritious"}, "idx": 0, "task_name": "openbookqa", "doc_id": 93, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When looking for good soil for plants, typically what is optimal?\nAnswer:", "continuation": " dry and sandy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 93, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When looking for good soil for plants, typically what is optimal?\nAnswer:", "continuation": " grainy and bitter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 93, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When looking for good soil for plants, typically what is optimal?\nAnswer:", "continuation": " compact and hard"}, "idx": 3, "task_name": "openbookqa", "doc_id": 93, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals are drawn to\nAnswer:", "continuation": " gold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 94, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals are drawn to\nAnswer:", "continuation": " houses"}, "idx": 1, "task_name": "openbookqa", "doc_id": 94, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals are drawn to\nAnswer:", "continuation": " feeders"}, "idx": 2, "task_name": "openbookqa", "doc_id": 94, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals are drawn to\nAnswer:", "continuation": " Carbon Dioxide"}, "idx": 3, "task_name": "openbookqa", "doc_id": 94, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Squirrels spend their fall\nAnswer:", "continuation": " looking for pretty leaves to collect"}, "idx": 0, "task_name": "openbookqa", "doc_id": 95, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Squirrels spend their fall\nAnswer:", "continuation": " stockpiling rocks for fighting in the winter"}, "idx": 1, "task_name": "openbookqa", "doc_id": 95, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Squirrels spend their fall\nAnswer:", "continuation": " stockpiling pecans for the frigid months"}, "idx": 2, "task_name": "openbookqa", "doc_id": 95, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Squirrels spend their fall\nAnswer:", "continuation": " collecting twigs to keep warm"}, "idx": 3, "task_name": "openbookqa", "doc_id": 95, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rainbows are always found after what?\nAnswer:", "continuation": " A fire"}, "idx": 0, "task_name": "openbookqa", "doc_id": 96, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rainbows are always found after what?\nAnswer:", "continuation": " A tornado"}, "idx": 1, "task_name": "openbookqa", "doc_id": 96, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rainbows are always found after what?\nAnswer:", "continuation": " Rainfall"}, "idx": 2, "task_name": "openbookqa", "doc_id": 96, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rainbows are always found after what?\nAnswer:", "continuation": " Cereal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 96, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Crop rotation has a positive impact on what?\nAnswer:", "continuation": " government mentality"}, "idx": 0, "task_name": "openbookqa", "doc_id": 97, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Crop rotation has a positive impact on what?\nAnswer:", "continuation": " dirt quality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 97, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Crop rotation has a positive impact on what?\nAnswer:", "continuation": " town economies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 97, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Crop rotation has a positive impact on what?\nAnswer:", "continuation": " crop watering"}, "idx": 3, "task_name": "openbookqa", "doc_id": 97, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the best method for detecting texture is\nAnswer:", "continuation": " rubbing it"}, "idx": 0, "task_name": "openbookqa", "doc_id": 98, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the best method for detecting texture is\nAnswer:", "continuation": " seeing it"}, "idx": 1, "task_name": "openbookqa", "doc_id": 98, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the best method for detecting texture is\nAnswer:", "continuation": " hearing it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 98, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the best method for detecting texture is\nAnswer:", "continuation": " tasting it"}, "idx": 3, "task_name": "openbookqa", "doc_id": 98, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cold-blooded animals are often\nAnswer:", "continuation": " fast"}, "idx": 0, "task_name": "openbookqa", "doc_id": 99, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cold-blooded animals are often\nAnswer:", "continuation": " large"}, "idx": 1, "task_name": "openbookqa", "doc_id": 99, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cold-blooded animals are often\nAnswer:", "continuation": " hairless"}, "idx": 2, "task_name": "openbookqa", "doc_id": 99, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cold-blooded animals are often\nAnswer:", "continuation": " slow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 99, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grey clouds can bring\nAnswer:", "continuation": " sunlight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grey clouds can bring\nAnswer:", "continuation": " falling water molecules"}, "idx": 1, "task_name": "openbookqa", "doc_id": 100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grey clouds can bring\nAnswer:", "continuation": " blooming flowers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grey clouds can bring\nAnswer:", "continuation": " drought conditions"}, "idx": 3, "task_name": "openbookqa", "doc_id": 100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is considered a predator?\nAnswer:", "continuation": " ant"}, "idx": 0, "task_name": "openbookqa", "doc_id": 101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is considered a predator?\nAnswer:", "continuation": " snake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is considered a predator?\nAnswer:", "continuation": " elephant"}, "idx": 2, "task_name": "openbookqa", "doc_id": 101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is considered a predator?\nAnswer:", "continuation": " giraffe"}, "idx": 3, "task_name": "openbookqa", "doc_id": 101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pollinators\nAnswer:", "continuation": " enable plants to continue flourishing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pollinators\nAnswer:", "continuation": " play an unimportant role in the reproduction process"}, "idx": 1, "task_name": "openbookqa", "doc_id": 102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pollinators\nAnswer:", "continuation": " are useless to plants"}, "idx": 2, "task_name": "openbookqa", "doc_id": 102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pollinators\nAnswer:", "continuation": " are considered unwanted pests"}, "idx": 3, "task_name": "openbookqa", "doc_id": 102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "continuation": " have their own dirt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "continuation": " have their own sunlight"}, "idx": 1, "task_name": "openbookqa", "doc_id": 103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "continuation": " have a lot of sand"}, "idx": 2, "task_name": "openbookqa", "doc_id": 103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "continuation": " contain their necessary nutrition"}, "idx": 3, "task_name": "openbookqa", "doc_id": 103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of characteristics are people not born with?\nAnswer:", "continuation": " genetics"}, "idx": 0, "task_name": "openbookqa", "doc_id": 104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of characteristics are people not born with?\nAnswer:", "continuation": " skills"}, "idx": 1, "task_name": "openbookqa", "doc_id": 104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of characteristics are people not born with?\nAnswer:", "continuation": " physical attributes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of characteristics are people not born with?\nAnswer:", "continuation": " height"}, "idx": 3, "task_name": "openbookqa", "doc_id": 104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A spinning object is used to make\nAnswer:", "continuation": " steam"}, "idx": 0, "task_name": "openbookqa", "doc_id": 105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A spinning object is used to make\nAnswer:", "continuation": " heat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A spinning object is used to make\nAnswer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A spinning object is used to make\nAnswer:", "continuation": " electricity"}, "idx": 3, "task_name": "openbookqa", "doc_id": 105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: One of the negative consequences of offshore oil platforms is\nAnswer:", "continuation": " evaporation of the surrounding water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: One of the negative consequences of offshore oil platforms is\nAnswer:", "continuation": " discharge of liquid petroleum in the surrounding sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: One of the negative consequences of offshore oil platforms is\nAnswer:", "continuation": " improvement in the conditions of sea life"}, "idx": 2, "task_name": "openbookqa", "doc_id": 106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: One of the negative consequences of offshore oil platforms is\nAnswer:", "continuation": " increase in the birthrate of sea birds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "continuation": " 1000"}, "idx": 0, "task_name": "openbookqa", "doc_id": 107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "continuation": " 250"}, "idx": 1, "task_name": "openbookqa", "doc_id": 107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "continuation": " 5000"}, "idx": 2, "task_name": "openbookqa", "doc_id": 107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "continuation": " 300"}, "idx": 3, "task_name": "openbookqa", "doc_id": 107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "continuation": " getting a new child"}, "idx": 0, "task_name": "openbookqa", "doc_id": 108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "continuation": " yelling at the mud"}, "idx": 1, "task_name": "openbookqa", "doc_id": 108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "continuation": " asking the child to stop"}, "idx": 2, "task_name": "openbookqa", "doc_id": 108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "continuation": " turning on the hose"}, "idx": 3, "task_name": "openbookqa", "doc_id": 108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthquakes\nAnswer:", "continuation": " only happen in California"}, "idx": 0, "task_name": "openbookqa", "doc_id": 109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthquakes\nAnswer:", "continuation": " cause solar and lunar eclipses"}, "idx": 1, "task_name": "openbookqa", "doc_id": 109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthquakes\nAnswer:", "continuation": " will break your vases"}, "idx": 2, "task_name": "openbookqa", "doc_id": 109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthquakes\nAnswer:", "continuation": " make bridges much safer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A seismograph can accurately describe\nAnswer:", "continuation": " how rough the footing will be"}, "idx": 0, "task_name": "openbookqa", "doc_id": 110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A seismograph can accurately describe\nAnswer:", "continuation": " how bad the weather will be"}, "idx": 1, "task_name": "openbookqa", "doc_id": 110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A seismograph can accurately describe\nAnswer:", "continuation": " how stable the ground will be"}, "idx": 2, "task_name": "openbookqa", "doc_id": 110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A seismograph can accurately describe\nAnswer:", "continuation": " how shaky the horse will be"}, "idx": 3, "task_name": "openbookqa", "doc_id": 110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "continuation": " the sun is always bright"}, "idx": 0, "task_name": "openbookqa", "doc_id": 111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "continuation": " the moon is brighter than stars"}, "idx": 1, "task_name": "openbookqa", "doc_id": 111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "continuation": " the moon is brighter than a floodlight"}, "idx": 2, "task_name": "openbookqa", "doc_id": 111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "continuation": " the sun is darker than the moon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant needs a specific climate to grow and\nAnswer:", "continuation": " wither"}, "idx": 0, "task_name": "openbookqa", "doc_id": 112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant needs a specific climate to grow and\nAnswer:", "continuation": " persist"}, "idx": 1, "task_name": "openbookqa", "doc_id": 112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant needs a specific climate to grow and\nAnswer:", "continuation": " die"}, "idx": 2, "task_name": "openbookqa", "doc_id": 112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant needs a specific climate to grow and\nAnswer:", "continuation": " decay"}, "idx": 3, "task_name": "openbookqa", "doc_id": 112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Banging on a drum causes\nAnswer:", "continuation": " music to be loud"}, "idx": 0, "task_name": "openbookqa", "doc_id": 113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Banging on a drum causes\nAnswer:", "continuation": " music to be appealing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Banging on a drum causes\nAnswer:", "continuation": " reverberations to strike the eardrum"}, "idx": 2, "task_name": "openbookqa", "doc_id": 113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Banging on a drum causes\nAnswer:", "continuation": " concerts to sell out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What may have been formed by a volcano?\nAnswer:", "continuation": " Mt. McKinley"}, "idx": 0, "task_name": "openbookqa", "doc_id": 114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What may have been formed by a volcano?\nAnswer:", "continuation": " Lake Pontchartrain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What may have been formed by a volcano?\nAnswer:", "continuation": " The great lakes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What may have been formed by a volcano?\nAnswer:", "continuation": " Niagara Falls"}, "idx": 3, "task_name": "openbookqa", "doc_id": 114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "continuation": " old"}, "idx": 0, "task_name": "openbookqa", "doc_id": 115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "continuation": " dead"}, "idx": 1, "task_name": "openbookqa", "doc_id": 115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "continuation": " in an egg"}, "idx": 2, "task_name": "openbookqa", "doc_id": 115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "continuation": " big"}, "idx": 3, "task_name": "openbookqa", "doc_id": 115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " basements to flood due to weather"}, "idx": 0, "task_name": "openbookqa", "doc_id": 116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " small birds to kill large birds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " waterfalls to flow backwards"}, "idx": 2, "task_name": "openbookqa", "doc_id": 116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " stones to weather down to pebbles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited characteristics\nAnswer:", "continuation": " include mice being able to navigate a maze"}, "idx": 0, "task_name": "openbookqa", "doc_id": 117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited characteristics\nAnswer:", "continuation": " include learning to sit on command"}, "idx": 1, "task_name": "openbookqa", "doc_id": 117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited characteristics\nAnswer:", "continuation": " include dolphins doing tricks for their trainers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Inherited characteristics\nAnswer:", "continuation": " include spots on a ladybug"}, "idx": 3, "task_name": "openbookqa", "doc_id": 117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How could we determine approximately how far a bird is from the ground?\nAnswer:", "continuation": " Measure the altitude of the bird using a reference point, such as a tall building."}, "idx": 0, "task_name": "openbookqa", "doc_id": 118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How could we determine approximately how far a bird is from the ground?\nAnswer:", "continuation": " Identify the species of bird"}, "idx": 1, "task_name": "openbookqa", "doc_id": 118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How could we determine approximately how far a bird is from the ground?\nAnswer:", "continuation": " Ask the bird how high it was when it returns back to earth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How could we determine approximately how far a bird is from the ground?\nAnswer:", "continuation": " Measure the bird's mass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Endangered pandas are sometimes\nAnswer:", "continuation": " accidentally dropped into volcanoes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Endangered pandas are sometimes\nAnswer:", "continuation": " confined to enclosures to be viewed by the public"}, "idx": 1, "task_name": "openbookqa", "doc_id": 119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Endangered pandas are sometimes\nAnswer:", "continuation": " found eating corn in the middle of North America"}, "idx": 2, "task_name": "openbookqa", "doc_id": 119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Endangered pandas are sometimes\nAnswer:", "continuation": " made into delicious rare steaks"}, "idx": 3, "task_name": "openbookqa", "doc_id": 119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Algae can be found in\nAnswer:", "continuation": " reservoir"}, "idx": 0, "task_name": "openbookqa", "doc_id": 120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Algae can be found in\nAnswer:", "continuation": " meat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Algae can be found in\nAnswer:", "continuation": " street"}, "idx": 2, "task_name": "openbookqa", "doc_id": 120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Algae can be found in\nAnswer:", "continuation": " tree"}, "idx": 3, "task_name": "openbookqa", "doc_id": 120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "continuation": " a campfire toasts bread"}, "idx": 0, "task_name": "openbookqa", "doc_id": 121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "continuation": " a microwave heats soup"}, "idx": 1, "task_name": "openbookqa", "doc_id": 121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "continuation": " a fire burns paper"}, "idx": 2, "task_name": "openbookqa", "doc_id": 121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "continuation": " a small oven works"}, "idx": 3, "task_name": "openbookqa", "doc_id": 121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "continuation": " mimicking other things"}, "idx": 0, "task_name": "openbookqa", "doc_id": 122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "continuation": " melting into sand"}, "idx": 1, "task_name": "openbookqa", "doc_id": 122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "continuation": " creating new homes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "continuation": " mocking other fish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does someone do when creating music?\nAnswer:", "continuation": " hit a toy baseball with a bat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does someone do when creating music?\nAnswer:", "continuation": " shake a baby rattle with your hand"}, "idx": 1, "task_name": "openbookqa", "doc_id": 123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does someone do when creating music?\nAnswer:", "continuation": " bang the wall with your fist"}, "idx": 2, "task_name": "openbookqa", "doc_id": 123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does someone do when creating music?\nAnswer:", "continuation": " pluck the strings of a fingerboard with your fingers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "continuation": " telescope views"}, "idx": 0, "task_name": "openbookqa", "doc_id": 124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "continuation": " water level fluctuations"}, "idx": 1, "task_name": "openbookqa", "doc_id": 124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "continuation": " animal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "continuation": " plant harvesting"}, "idx": 3, "task_name": "openbookqa", "doc_id": 124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "continuation": " baby oil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "continuation": " corn syrup or"}, "idx": 1, "task_name": "openbookqa", "doc_id": 125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "continuation": " milk"}, "idx": 2, "task_name": "openbookqa", "doc_id": 125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "continuation": " honey"}, "idx": 3, "task_name": "openbookqa", "doc_id": 125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis can be performed by\nAnswer:", "continuation": " a cabbage cell"}, "idx": 0, "task_name": "openbookqa", "doc_id": 126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis can be performed by\nAnswer:", "continuation": " a bee cell"}, "idx": 1, "task_name": "openbookqa", "doc_id": 126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis can be performed by\nAnswer:", "continuation": " a bear cell"}, "idx": 2, "task_name": "openbookqa", "doc_id": 126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis can be performed by\nAnswer:", "continuation": " a cat cell"}, "idx": 3, "task_name": "openbookqa", "doc_id": 126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "continuation": " reverse"}, "idx": 0, "task_name": "openbookqa", "doc_id": 127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "continuation": " inverse"}, "idx": 1, "task_name": "openbookqa", "doc_id": 127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "continuation": " equal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "continuation": " direct"}, "idx": 3, "task_name": "openbookqa", "doc_id": 127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: all cells use cellular respiration to\nAnswer:", "continuation": " photosynthesize"}, "idx": 0, "task_name": "openbookqa", "doc_id": 128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: all cells use cellular respiration to\nAnswer:", "continuation": " release waste"}, "idx": 1, "task_name": "openbookqa", "doc_id": 128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: all cells use cellular respiration to\nAnswer:", "continuation": " perform meiosis"}, "idx": 2, "task_name": "openbookqa", "doc_id": 128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: all cells use cellular respiration to\nAnswer:", "continuation": " release energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "continuation": " reproductive purposes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "continuation": " viewing sounds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "continuation": " sensing views"}, "idx": 2, "task_name": "openbookqa", "doc_id": 129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "continuation": " sensing tastes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "continuation": " breaks for birds on the road"}, "idx": 0, "task_name": "openbookqa", "doc_id": 130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "continuation": " given a large breakfast"}, "idx": 1, "task_name": "openbookqa", "doc_id": 130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "continuation": " eats a few corn cobs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "continuation": " given two apples to watch"}, "idx": 3, "task_name": "openbookqa", "doc_id": 130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "continuation": " the woman makes more money"}, "idx": 0, "task_name": "openbookqa", "doc_id": 131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "continuation": " the woman makes the same amount of money"}, "idx": 1, "task_name": "openbookqa", "doc_id": 131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "continuation": " the woman spends more money"}, "idx": 2, "task_name": "openbookqa", "doc_id": 131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "continuation": " the woman makes less money"}, "idx": 3, "task_name": "openbookqa", "doc_id": 131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A desert environment is\nAnswer:", "continuation": " dry, grass covered, and humid"}, "idx": 0, "task_name": "openbookqa", "doc_id": 132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A desert environment is\nAnswer:", "continuation": " lush, green, and tropical"}, "idx": 1, "task_name": "openbookqa", "doc_id": 132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A desert environment is\nAnswer:", "continuation": " arid, parched, and sun-baked"}, "idx": 2, "task_name": "openbookqa", "doc_id": 132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A desert environment is\nAnswer:", "continuation": " dry, damp, and lush"}, "idx": 3, "task_name": "openbookqa", "doc_id": 132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seasons are caused by what rotating on its axis?\nAnswer:", "continuation": " Our Planet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seasons are caused by what rotating on its axis?\nAnswer:", "continuation": " The Atmosphere"}, "idx": 1, "task_name": "openbookqa", "doc_id": 133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seasons are caused by what rotating on its axis?\nAnswer:", "continuation": " The Equator"}, "idx": 2, "task_name": "openbookqa", "doc_id": 133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seasons are caused by what rotating on its axis?\nAnswer:", "continuation": " The Sun"}, "idx": 3, "task_name": "openbookqa", "doc_id": 133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is best an letting electricity pass through?\nAnswer:", "continuation": " tile flooring"}, "idx": 0, "task_name": "openbookqa", "doc_id": 134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is best an letting electricity pass through?\nAnswer:", "continuation": " human flesh"}, "idx": 1, "task_name": "openbookqa", "doc_id": 134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is best an letting electricity pass through?\nAnswer:", "continuation": " hockey stick"}, "idx": 2, "task_name": "openbookqa", "doc_id": 134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is best an letting electricity pass through?\nAnswer:", "continuation": " a steak knife"}, "idx": 3, "task_name": "openbookqa", "doc_id": 134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Asteroids crashing on planets can leave behind\nAnswer:", "continuation": " large, bowl-shaped cavities in the ground"}, "idx": 0, "task_name": "openbookqa", "doc_id": 135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Asteroids crashing on planets can leave behind\nAnswer:", "continuation": " aliens and foreign foods"}, "idx": 1, "task_name": "openbookqa", "doc_id": 135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Asteroids crashing on planets can leave behind\nAnswer:", "continuation": " small dents in the planet's core"}, "idx": 2, "task_name": "openbookqa", "doc_id": 135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Asteroids crashing on planets can leave behind\nAnswer:", "continuation": " lakes filled with salty water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the formula of the substance which best helps plants grow\nAnswer:", "continuation": " NH4"}, "idx": 0, "task_name": "openbookqa", "doc_id": 136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the formula of the substance which best helps plants grow\nAnswer:", "continuation": " C4H4"}, "idx": 1, "task_name": "openbookqa", "doc_id": 136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the formula of the substance which best helps plants grow\nAnswer:", "continuation": " CO2"}, "idx": 2, "task_name": "openbookqa", "doc_id": 136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the formula of the substance which best helps plants grow\nAnswer:", "continuation": " H2O"}, "idx": 3, "task_name": "openbookqa", "doc_id": 136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: You can experience a change of pressure when\nAnswer:", "continuation": " Yelling really loud"}, "idx": 0, "task_name": "openbookqa", "doc_id": 137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: You can experience a change of pressure when\nAnswer:", "continuation": " Soaring the skies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: You can experience a change of pressure when\nAnswer:", "continuation": " Going walking"}, "idx": 2, "task_name": "openbookqa", "doc_id": 137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: You can experience a change of pressure when\nAnswer:", "continuation": " riding a bike"}, "idx": 3, "task_name": "openbookqa", "doc_id": 137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small creek absorbing heat energy can result in\nAnswer:", "continuation": " the creek water getting colder"}, "idx": 0, "task_name": "openbookqa", "doc_id": 138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small creek absorbing heat energy can result in\nAnswer:", "continuation": " a parched creek bed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small creek absorbing heat energy can result in\nAnswer:", "continuation": " tributaries branching off from the creek"}, "idx": 2, "task_name": "openbookqa", "doc_id": 138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small creek absorbing heat energy can result in\nAnswer:", "continuation": " a runoff of extra water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Summertime  happens during June in all but which location?\nAnswer:", "continuation": " Australia"}, "idx": 0, "task_name": "openbookqa", "doc_id": 139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Summertime  happens during June in all but which location?\nAnswer:", "continuation": " in Canada"}, "idx": 1, "task_name": "openbookqa", "doc_id": 139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Summertime  happens during June in all but which location?\nAnswer:", "continuation": " United States"}, "idx": 2, "task_name": "openbookqa", "doc_id": 139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Summertime  happens during June in all but which location?\nAnswer:", "continuation": " Europe"}, "idx": 3, "task_name": "openbookqa", "doc_id": 139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It takes more water to fill a bathtub than a\nAnswer:", "continuation": " lake"}, "idx": 0, "task_name": "openbookqa", "doc_id": 140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It takes more water to fill a bathtub than a\nAnswer:", "continuation": " pool"}, "idx": 1, "task_name": "openbookqa", "doc_id": 140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It takes more water to fill a bathtub than a\nAnswer:", "continuation": " stomach"}, "idx": 2, "task_name": "openbookqa", "doc_id": 140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: It takes more water to fill a bathtub than a\nAnswer:", "continuation": " holding tank"}, "idx": 3, "task_name": "openbookqa", "doc_id": 140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The eighth month of the year is winter in\nAnswer:", "continuation": " Brazil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The eighth month of the year is winter in\nAnswer:", "continuation": " Indiana"}, "idx": 1, "task_name": "openbookqa", "doc_id": 141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The eighth month of the year is winter in\nAnswer:", "continuation": " London"}, "idx": 2, "task_name": "openbookqa", "doc_id": 141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The eighth month of the year is winter in\nAnswer:", "continuation": " Canada"}, "idx": 3, "task_name": "openbookqa", "doc_id": 141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: birds use their peckers to catch\nAnswer:", "continuation": " dogs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: birds use their peckers to catch\nAnswer:", "continuation": " a tan"}, "idx": 1, "task_name": "openbookqa", "doc_id": 142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: birds use their peckers to catch\nAnswer:", "continuation": " a ball"}, "idx": 2, "task_name": "openbookqa", "doc_id": 142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: birds use their peckers to catch\nAnswer:", "continuation": " bees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: transplanting seedling oaks has a positive impact on\nAnswer:", "continuation": " fuel costs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: transplanting seedling oaks has a positive impact on\nAnswer:", "continuation": " the economy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: transplanting seedling oaks has a positive impact on\nAnswer:", "continuation": " housing value"}, "idx": 2, "task_name": "openbookqa", "doc_id": 143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: transplanting seedling oaks has a positive impact on\nAnswer:", "continuation": " the environment"}, "idx": 3, "task_name": "openbookqa", "doc_id": 143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of hunting?\nAnswer:", "continuation": " humans throwing a spear through an animal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of hunting?\nAnswer:", "continuation": " humans chewing on boiled animal muscles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of hunting?\nAnswer:", "continuation": " humans gathering animals in a gate"}, "idx": 2, "task_name": "openbookqa", "doc_id": 144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of hunting?\nAnswer:", "continuation": " humans plucking fruit from a tree"}, "idx": 3, "task_name": "openbookqa", "doc_id": 144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "continuation": " paper"}, "idx": 0, "task_name": "openbookqa", "doc_id": 145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "continuation": " wood"}, "idx": 1, "task_name": "openbookqa", "doc_id": 145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "continuation": " stone"}, "idx": 2, "task_name": "openbookqa", "doc_id": 145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "continuation": " orange juice"}, "idx": 3, "task_name": "openbookqa", "doc_id": 145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "continuation": " Sweet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "continuation": " Salty"}, "idx": 1, "task_name": "openbookqa", "doc_id": 146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "continuation": " bitter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "continuation": " sour"}, "idx": 3, "task_name": "openbookqa", "doc_id": 146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth's four layers are comprised mainly of\nAnswer:", "continuation": " stone"}, "idx": 0, "task_name": "openbookqa", "doc_id": 147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth's four layers are comprised mainly of\nAnswer:", "continuation": " bacteria"}, "idx": 1, "task_name": "openbookqa", "doc_id": 147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth's four layers are comprised mainly of\nAnswer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth's four layers are comprised mainly of\nAnswer:", "continuation": " air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "continuation": " the freezing point"}, "idx": 0, "task_name": "openbookqa", "doc_id": 148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "continuation": " the prime point"}, "idx": 1, "task_name": "openbookqa", "doc_id": 148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "continuation": " the boiling point"}, "idx": 2, "task_name": "openbookqa", "doc_id": 148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "continuation": " the melting point"}, "idx": 3, "task_name": "openbookqa", "doc_id": 148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mammals give birth to live\nAnswer:", "continuation": " children"}, "idx": 0, "task_name": "openbookqa", "doc_id": 149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mammals give birth to live\nAnswer:", "continuation": " birds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mammals give birth to live\nAnswer:", "continuation": " fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Mammals give birth to live\nAnswer:", "continuation": " insects"}, "idx": 3, "task_name": "openbookqa", "doc_id": 149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "continuation": " it loses its magnetization"}, "idx": 0, "task_name": "openbookqa", "doc_id": 150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "continuation": " it loses its charge"}, "idx": 1, "task_name": "openbookqa", "doc_id": 150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "continuation": " it may become magnetized"}, "idx": 2, "task_name": "openbookqa", "doc_id": 150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "continuation": " it gains a charge"}, "idx": 3, "task_name": "openbookqa", "doc_id": 150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "continuation": " luck"}, "idx": 0, "task_name": "openbookqa", "doc_id": 151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "continuation": " humans"}, "idx": 1, "task_name": "openbookqa", "doc_id": 151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "continuation": " gold"}, "idx": 2, "task_name": "openbookqa", "doc_id": 151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "continuation": " inheritance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is recyclable?\nAnswer:", "continuation": " An Elephant"}, "idx": 0, "task_name": "openbookqa", "doc_id": 152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is recyclable?\nAnswer:", "continuation": " A school notebook"}, "idx": 1, "task_name": "openbookqa", "doc_id": 152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is recyclable?\nAnswer:", "continuation": " A boat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which is recyclable?\nAnswer:", "continuation": " A lake"}, "idx": 3, "task_name": "openbookqa", "doc_id": 152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "continuation": " it is a place to emigrate from"}, "idx": 0, "task_name": "openbookqa", "doc_id": 153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "continuation": " it is an ideal habitat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "continuation": " it is an unsustainable habitat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "continuation": " it is a thriving abode"}, "idx": 3, "task_name": "openbookqa", "doc_id": 153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "continuation": " water and soda"}, "idx": 0, "task_name": "openbookqa", "doc_id": 154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "continuation": " water and oil"}, "idx": 1, "task_name": "openbookqa", "doc_id": 154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "continuation": " sand and rocks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "continuation": " salt and bark"}, "idx": 3, "task_name": "openbookqa", "doc_id": 154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "continuation": " spider bites"}, "idx": 0, "task_name": "openbookqa", "doc_id": 155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "continuation": " vomiting"}, "idx": 1, "task_name": "openbookqa", "doc_id": 155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "continuation": " placebo effect"}, "idx": 2, "task_name": "openbookqa", "doc_id": 155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "continuation": " dance fever"}, "idx": 3, "task_name": "openbookqa", "doc_id": 155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Lightning may lead to\nAnswer:", "continuation": " damage to local foliage"}, "idx": 0, "task_name": "openbookqa", "doc_id": 156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Lightning may lead to\nAnswer:", "continuation": " rainbows across the sky"}, "idx": 1, "task_name": "openbookqa", "doc_id": 156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Lightning may lead to\nAnswer:", "continuation": " growth of local flora"}, "idx": 2, "task_name": "openbookqa", "doc_id": 156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Lightning may lead to\nAnswer:", "continuation": " firefighters getting the night off"}, "idx": 3, "task_name": "openbookqa", "doc_id": 156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When would a nocturnal predator most likely hunt?\nAnswer:", "continuation": " 5 p.m."}, "idx": 0, "task_name": "openbookqa", "doc_id": 157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When would a nocturnal predator most likely hunt?\nAnswer:", "continuation": " 12 p.m."}, "idx": 1, "task_name": "openbookqa", "doc_id": 157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When would a nocturnal predator most likely hunt?\nAnswer:", "continuation": " 3 a.m."}, "idx": 2, "task_name": "openbookqa", "doc_id": 157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When would a nocturnal predator most likely hunt?\nAnswer:", "continuation": " 10 a.m."}, "idx": 3, "task_name": "openbookqa", "doc_id": 157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where water be located in its gas form?\nAnswer:", "continuation": " inside a disc golf driver"}, "idx": 0, "task_name": "openbookqa", "doc_id": 158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where water be located in its gas form?\nAnswer:", "continuation": " inside of a brass pipe"}, "idx": 1, "task_name": "openbookqa", "doc_id": 158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where water be located in its gas form?\nAnswer:", "continuation": " a mile up in the sky"}, "idx": 2, "task_name": "openbookqa", "doc_id": 158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where water be located in its gas form?\nAnswer:", "continuation": " inside a leather baseball"}, "idx": 3, "task_name": "openbookqa", "doc_id": 158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Green parts of a life form absorb\nAnswer:", "continuation": " carbon dioxide"}, "idx": 0, "task_name": "openbookqa", "doc_id": 159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Green parts of a life form absorb\nAnswer:", "continuation": " light"}, "idx": 1, "task_name": "openbookqa", "doc_id": 159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Green parts of a life form absorb\nAnswer:", "continuation": " oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Green parts of a life form absorb\nAnswer:", "continuation": " water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a new species of predator joins a community\nAnswer:", "continuation": " the new species will become herbivores"}, "idx": 0, "task_name": "openbookqa", "doc_id": 160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a new species of predator joins a community\nAnswer:", "continuation": " prey will experience an increase in population"}, "idx": 1, "task_name": "openbookqa", "doc_id": 160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a new species of predator joins a community\nAnswer:", "continuation": " prey will experience a drop in population"}, "idx": 2, "task_name": "openbookqa", "doc_id": 160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a new species of predator joins a community\nAnswer:", "continuation": " the old species will die out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "continuation": " It's dead"}, "idx": 0, "task_name": "openbookqa", "doc_id": 161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "continuation": " For water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "continuation": " For food"}, "idx": 2, "task_name": "openbookqa", "doc_id": 161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "continuation": " For sun"}, "idx": 3, "task_name": "openbookqa", "doc_id": 161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "continuation": " weasel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "continuation": " snow fox"}, "idx": 1, "task_name": "openbookqa", "doc_id": 162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "continuation": " wolf"}, "idx": 2, "task_name": "openbookqa", "doc_id": 162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "continuation": " polar bear"}, "idx": 3, "task_name": "openbookqa", "doc_id": 162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dairy is a source of\nAnswer:", "continuation": " a vitamin that prevents blood loss"}, "idx": 0, "task_name": "openbookqa", "doc_id": 163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dairy is a source of\nAnswer:", "continuation": " a vitamin that treats amino acid deficiency"}, "idx": 1, "task_name": "openbookqa", "doc_id": 163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dairy is a source of\nAnswer:", "continuation": " a group of fat-soluble secosteroids"}, "idx": 2, "task_name": "openbookqa", "doc_id": 163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dairy is a source of\nAnswer:", "continuation": " a vitamin that helps treat liver problems"}, "idx": 3, "task_name": "openbookqa", "doc_id": 163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Atomic 26 is drawn to a device, it could be\nAnswer:", "continuation": " magnetized"}, "idx": 0, "task_name": "openbookqa", "doc_id": 164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Atomic 26 is drawn to a device, it could be\nAnswer:", "continuation": " Na"}, "idx": 1, "task_name": "openbookqa", "doc_id": 164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Atomic 26 is drawn to a device, it could be\nAnswer:", "continuation": " compass"}, "idx": 2, "task_name": "openbookqa", "doc_id": 164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Atomic 26 is drawn to a device, it could be\nAnswer:", "continuation": " K"}, "idx": 3, "task_name": "openbookqa", "doc_id": 164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: why do tadpoles change into frogs?\nAnswer:", "continuation": " tadpoles change to different animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: why do tadpoles change into frogs?\nAnswer:", "continuation": " tadpoles are really just fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: why do tadpoles change into frogs?\nAnswer:", "continuation": " they are young frogs still growing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: why do tadpoles change into frogs?\nAnswer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do cows eat?\nAnswer:", "continuation": " Chickpeas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do cows eat?\nAnswer:", "continuation": " Chocolate"}, "idx": 1, "task_name": "openbookqa", "doc_id": 166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do cows eat?\nAnswer:", "continuation": " Steak"}, "idx": 2, "task_name": "openbookqa", "doc_id": 166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do cows eat?\nAnswer:", "continuation": " Poultry"}, "idx": 3, "task_name": "openbookqa", "doc_id": 166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which object conducts electricity?\nAnswer:", "continuation": " Window"}, "idx": 0, "task_name": "openbookqa", "doc_id": 167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which object conducts electricity?\nAnswer:", "continuation": " Rubik's Cube"}, "idx": 1, "task_name": "openbookqa", "doc_id": 167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which object conducts electricity?\nAnswer:", "continuation": " Ship Anchor"}, "idx": 2, "task_name": "openbookqa", "doc_id": 167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which object conducts electricity?\nAnswer:", "continuation": " Boulder"}, "idx": 3, "task_name": "openbookqa", "doc_id": 167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Glucose travels\nAnswer:", "continuation": " from roots to leaves of a daffodil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Glucose travels\nAnswer:", "continuation": " from a rose's leaves to the atmosphere"}, "idx": 1, "task_name": "openbookqa", "doc_id": 168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Glucose travels\nAnswer:", "continuation": " from a daisy's leaves into it's underground support system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Glucose travels\nAnswer:", "continuation": " from the sun to a sunflower's buds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " only happens in the summer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " is like nature's disappearing water trick"}, "idx": 1, "task_name": "openbookqa", "doc_id": 169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " is caused by snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " involves the disappearance of sunlight"}, "idx": 3, "task_name": "openbookqa", "doc_id": 169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "continuation": " air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "continuation": " Corn"}, "idx": 1, "task_name": "openbookqa", "doc_id": 170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "continuation": " Runoff"}, "idx": 2, "task_name": "openbookqa", "doc_id": 170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "continuation": " farmers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "continuation": " friction"}, "idx": 0, "task_name": "openbookqa", "doc_id": 171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "continuation": " metal"}, "idx": 1, "task_name": "openbookqa", "doc_id": 171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "continuation": " wishing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "continuation": " magic"}, "idx": 3, "task_name": "openbookqa", "doc_id": 171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a duck like to live?\nAnswer:", "continuation": " the Sahara"}, "idx": 0, "task_name": "openbookqa", "doc_id": 172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a duck like to live?\nAnswer:", "continuation": " Antarctica"}, "idx": 1, "task_name": "openbookqa", "doc_id": 172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a duck like to live?\nAnswer:", "continuation": " the Appalachian mountains"}, "idx": 2, "task_name": "openbookqa", "doc_id": 172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a duck like to live?\nAnswer:", "continuation": " Death Valley"}, "idx": 3, "task_name": "openbookqa", "doc_id": 172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do tuna eat?\nAnswer:", "continuation": " Atlantic menhaden"}, "idx": 0, "task_name": "openbookqa", "doc_id": 173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do tuna eat?\nAnswer:", "continuation": " Swedish fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do tuna eat?\nAnswer:", "continuation": " gummy fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What do tuna eat?\nAnswer:", "continuation": " laminariales"}, "idx": 3, "task_name": "openbookqa", "doc_id": 173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "continuation": " flowers blooming"}, "idx": 0, "task_name": "openbookqa", "doc_id": 174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "continuation": " grass turning brown"}, "idx": 1, "task_name": "openbookqa", "doc_id": 174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "continuation": " trees growing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "continuation": " blossoms blooming"}, "idx": 3, "task_name": "openbookqa", "doc_id": 174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: is it normal for an adult animal to lay eggs?\nAnswer:", "continuation": " it has never happened"}, "idx": 0, "task_name": "openbookqa", "doc_id": 175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: is it normal for an adult animal to lay eggs?\nAnswer:", "continuation": " yes it is standard"}, "idx": 1, "task_name": "openbookqa", "doc_id": 175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: is it normal for an adult animal to lay eggs?\nAnswer:", "continuation": " it is abnormal and weird"}, "idx": 2, "task_name": "openbookqa", "doc_id": 175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: is it normal for an adult animal to lay eggs?\nAnswer:", "continuation": " all of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog is overweight\nAnswer:", "continuation": " add more fat to their diet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog is overweight\nAnswer:", "continuation": " cut back their caloric intake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog is overweight\nAnswer:", "continuation": " let them sleep more"}, "idx": 2, "task_name": "openbookqa", "doc_id": 176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog is overweight\nAnswer:", "continuation": " increase their caloric intake"}, "idx": 3, "task_name": "openbookqa", "doc_id": 176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Eyes allow humans\nAnswer:", "continuation": " to detect when a traffic light changes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Eyes allow humans\nAnswer:", "continuation": " detect sour flavors in candy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Eyes allow humans\nAnswer:", "continuation": " hear music at concerts"}, "idx": 2, "task_name": "openbookqa", "doc_id": 177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Eyes allow humans\nAnswer:", "continuation": " detect acrid odors in the air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Organisms covered by layers of sediment\nAnswer:", "continuation": " become fossils over night"}, "idx": 0, "task_name": "openbookqa", "doc_id": 178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Organisms covered by layers of sediment\nAnswer:", "continuation": " may end up reanimated over time"}, "idx": 1, "task_name": "openbookqa", "doc_id": 178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Organisms covered by layers of sediment\nAnswer:", "continuation": " develop characteristics for survival"}, "idx": 2, "task_name": "openbookqa", "doc_id": 178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Organisms covered by layers of sediment\nAnswer:", "continuation": " may end up fueling a car"}, "idx": 3, "task_name": "openbookqa", "doc_id": 178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The winter solstice is on December 21st in the\nAnswer:", "continuation": " counties"}, "idx": 0, "task_name": "openbookqa", "doc_id": 179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The winter solstice is on December 21st in the\nAnswer:", "continuation": " north of equator"}, "idx": 1, "task_name": "openbookqa", "doc_id": 179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The winter solstice is on December 21st in the\nAnswer:", "continuation": " states"}, "idx": 2, "task_name": "openbookqa", "doc_id": 179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The winter solstice is on December 21st in the\nAnswer:", "continuation": " southern hemisphere"}, "idx": 3, "task_name": "openbookqa", "doc_id": 179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "continuation": " bleaching"}, "idx": 0, "task_name": "openbookqa", "doc_id": 180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "continuation": " burning"}, "idx": 1, "task_name": "openbookqa", "doc_id": 180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "continuation": " fossilization"}, "idx": 2, "task_name": "openbookqa", "doc_id": 180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "continuation": " drying"}, "idx": 3, "task_name": "openbookqa", "doc_id": 180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: were there fossil fuels in the ground when humans evolved?\nAnswer:", "continuation": " this was only created by humans"}, "idx": 0, "task_name": "openbookqa", "doc_id": 181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: were there fossil fuels in the ground when humans evolved?\nAnswer:", "continuation": " humans predate fossil fuel formation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: were there fossil fuels in the ground when humans evolved?\nAnswer:", "continuation": " significant supplies accumulated prior"}, "idx": 2, "task_name": "openbookqa", "doc_id": 181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: were there fossil fuels in the ground when humans evolved?\nAnswer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "continuation": " some water is around it"}, "idx": 0, "task_name": "openbookqa", "doc_id": 182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "continuation": " water rained on it"}, "idx": 1, "task_name": "openbookqa", "doc_id": 182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "continuation": " natural waters weathered it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "continuation": " a pool was opened"}, "idx": 3, "task_name": "openbookqa", "doc_id": 182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of useful product can be made from the moving winds?\nAnswer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of useful product can be made from the moving winds?\nAnswer:", "continuation": " bananas"}, "idx": 1, "task_name": "openbookqa", "doc_id": 183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of useful product can be made from the moving winds?\nAnswer:", "continuation": " electricity"}, "idx": 2, "task_name": "openbookqa", "doc_id": 183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What type of useful product can be made from the moving winds?\nAnswer:", "continuation": " metal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sides of the canyon are\nAnswer:", "continuation": " metal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sides of the canyon are\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sides of the canyon are\nAnswer:", "continuation": " rivers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sides of the canyon are\nAnswer:", "continuation": " stone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these people would have the worst air quality at their residence?\nAnswer:", "continuation": " a man who lives next to a landfill"}, "idx": 0, "task_name": "openbookqa", "doc_id": 185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these people would have the worst air quality at their residence?\nAnswer:", "continuation": " a man who lives in a city with the best air quality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these people would have the worst air quality at their residence?\nAnswer:", "continuation": " none of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these people would have the worst air quality at their residence?\nAnswer:", "continuation": " a man who lives in a great suburb"}, "idx": 3, "task_name": "openbookqa", "doc_id": 185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a stopwatch used for?\nAnswer:", "continuation": " to rewind 5 minutes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a stopwatch used for?\nAnswer:", "continuation": " to tell what will happen 5 minutes from now"}, "idx": 1, "task_name": "openbookqa", "doc_id": 186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a stopwatch used for?\nAnswer:", "continuation": " to voice the time"}, "idx": 2, "task_name": "openbookqa", "doc_id": 186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is a stopwatch used for?\nAnswer:", "continuation": " to measure minutes and hours"}, "idx": 3, "task_name": "openbookqa", "doc_id": 186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As a drought worsens the level at an aquifer will\nAnswer:", "continuation": " stay the same"}, "idx": 0, "task_name": "openbookqa", "doc_id": 187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As a drought worsens the level at an aquifer will\nAnswer:", "continuation": " fluctuate wildly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As a drought worsens the level at an aquifer will\nAnswer:", "continuation": " decrease"}, "idx": 2, "task_name": "openbookqa", "doc_id": 187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: As a drought worsens the level at an aquifer will\nAnswer:", "continuation": " increase"}, "idx": 3, "task_name": "openbookqa", "doc_id": 187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind frequently helps transport from one place to another\nAnswer:", "continuation": " marble statues"}, "idx": 0, "task_name": "openbookqa", "doc_id": 188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind frequently helps transport from one place to another\nAnswer:", "continuation": " molten magma"}, "idx": 1, "task_name": "openbookqa", "doc_id": 188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind frequently helps transport from one place to another\nAnswer:", "continuation": " subterranean termites"}, "idx": 2, "task_name": "openbookqa", "doc_id": 188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind frequently helps transport from one place to another\nAnswer:", "continuation": " exposed topsoil"}, "idx": 3, "task_name": "openbookqa", "doc_id": 188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "continuation": " bike helmet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "continuation": " avoiding accidents"}, "idx": 1, "task_name": "openbookqa", "doc_id": 189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "continuation": " friction"}, "idx": 2, "task_name": "openbookqa", "doc_id": 189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "continuation": " gearing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of spices have\nAnswer:", "continuation": " crystals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of spices have\nAnswer:", "continuation": " feathers"}, "idx": 1, "task_name": "openbookqa", "doc_id": 190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of spices have\nAnswer:", "continuation": " cell walls"}, "idx": 2, "task_name": "openbookqa", "doc_id": 190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of spices have\nAnswer:", "continuation": " craters"}, "idx": 3, "task_name": "openbookqa", "doc_id": 190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Conservation\nAnswer:", "continuation": " leads to longer drought of resources"}, "idx": 0, "task_name": "openbookqa", "doc_id": 191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Conservation\nAnswer:", "continuation": " leads to longer availability of resources"}, "idx": 1, "task_name": "openbookqa", "doc_id": 191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Conservation\nAnswer:", "continuation": " leads to more consumption"}, "idx": 2, "task_name": "openbookqa", "doc_id": 191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Conservation\nAnswer:", "continuation": " leads to short supply of resources"}, "idx": 3, "task_name": "openbookqa", "doc_id": 191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tree is not the habitat of a\nAnswer:", "continuation": " squirrel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tree is not the habitat of a\nAnswer:", "continuation": " woodpecker"}, "idx": 1, "task_name": "openbookqa", "doc_id": 192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tree is not the habitat of a\nAnswer:", "continuation": " monkey"}, "idx": 2, "task_name": "openbookqa", "doc_id": 192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A tree is not the habitat of a\nAnswer:", "continuation": " lion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion could lead to\nAnswer:", "continuation": " a change in the direction of a stream"}, "idx": 0, "task_name": "openbookqa", "doc_id": 193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion could lead to\nAnswer:", "continuation": " a change in ocean temperatures"}, "idx": 1, "task_name": "openbookqa", "doc_id": 193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion could lead to\nAnswer:", "continuation": " an increase in rainy weather"}, "idx": 2, "task_name": "openbookqa", "doc_id": 193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion could lead to\nAnswer:", "continuation": " an increase in plants and animals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A hemisphere experiences summer when\nAnswer:", "continuation": " it's tilted towards Jupiter"}, "idx": 0, "task_name": "openbookqa", "doc_id": 194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A hemisphere experiences summer when\nAnswer:", "continuation": " it's angled towards the moon"}, "idx": 1, "task_name": "openbookqa", "doc_id": 194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A hemisphere experiences summer when\nAnswer:", "continuation": " it's angled towards the largest star in the solar system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A hemisphere experiences summer when\nAnswer:", "continuation": " it spins counter clockwise on Earth's axis"}, "idx": 3, "task_name": "openbookqa", "doc_id": 194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What date is the amount of daylight minimized\nAnswer:", "continuation": " Jul 4th"}, "idx": 0, "task_name": "openbookqa", "doc_id": 195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What date is the amount of daylight minimized\nAnswer:", "continuation": " Feb 29th"}, "idx": 1, "task_name": "openbookqa", "doc_id": 195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What date is the amount of daylight minimized\nAnswer:", "continuation": " May 3rd"}, "idx": 2, "task_name": "openbookqa", "doc_id": 195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What date is the amount of daylight minimized\nAnswer:", "continuation": " Sep 1st"}, "idx": 3, "task_name": "openbookqa", "doc_id": 195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which term is involved with protection by skin?\nAnswer:", "continuation": " Eucerin pH5 range"}, "idx": 0, "task_name": "openbookqa", "doc_id": 196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which term is involved with protection by skin?\nAnswer:", "continuation": " Sagittal plane"}, "idx": 1, "task_name": "openbookqa", "doc_id": 196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which term is involved with protection by skin?\nAnswer:", "continuation": " pyogenic vibrio"}, "idx": 2, "task_name": "openbookqa", "doc_id": 196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which term is involved with protection by skin?\nAnswer:", "continuation": " popliteus"}, "idx": 3, "task_name": "openbookqa", "doc_id": 196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The reason Earth is so sturdy is because\nAnswer:", "continuation": " It is made from rock"}, "idx": 0, "task_name": "openbookqa", "doc_id": 197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The reason Earth is so sturdy is because\nAnswer:", "continuation": " It eats three meals a day"}, "idx": 1, "task_name": "openbookqa", "doc_id": 197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The reason Earth is so sturdy is because\nAnswer:", "continuation": " It has a loving family"}, "idx": 2, "task_name": "openbookqa", "doc_id": 197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The reason Earth is so sturdy is because\nAnswer:", "continuation": " It is made from metal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant left in the dark\nAnswer:", "continuation": " produces fruit"}, "idx": 0, "task_name": "openbookqa", "doc_id": 198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant left in the dark\nAnswer:", "continuation": " grows faster"}, "idx": 1, "task_name": "openbookqa", "doc_id": 198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant left in the dark\nAnswer:", "continuation": " fails to grow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant left in the dark\nAnswer:", "continuation": " gets greener"}, "idx": 3, "task_name": "openbookqa", "doc_id": 198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "continuation": " heat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "continuation": " metal"}, "idx": 1, "task_name": "openbookqa", "doc_id": 199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "continuation": " lithium-ion"}, "idx": 2, "task_name": "openbookqa", "doc_id": 199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "continuation": " plastic"}, "idx": 3, "task_name": "openbookqa", "doc_id": 199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "continuation": " axial planetary rotation"}, "idx": 0, "task_name": "openbookqa", "doc_id": 200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "continuation": " comets"}, "idx": 1, "task_name": "openbookqa", "doc_id": 200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "continuation": " planetary formation"}, "idx": 2, "task_name": "openbookqa", "doc_id": 200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "continuation": " the sun's rays"}, "idx": 3, "task_name": "openbookqa", "doc_id": 200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which source provides the safest water?\nAnswer:", "continuation": " River"}, "idx": 0, "task_name": "openbookqa", "doc_id": 201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which source provides the safest water?\nAnswer:", "continuation": " Sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which source provides the safest water?\nAnswer:", "continuation": " Ocean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which source provides the safest water?\nAnswer:", "continuation": " Rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "continuation": " spread out some fur"}, "idx": 0, "task_name": "openbookqa", "doc_id": 202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "continuation": " has black on his ears"}, "idx": 1, "task_name": "openbookqa", "doc_id": 202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "continuation": " passed down inherited characteristics"}, "idx": 2, "task_name": "openbookqa", "doc_id": 202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "continuation": " is the same size as the mother"}, "idx": 3, "task_name": "openbookqa", "doc_id": 202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis means plants are unable to\nAnswer:", "continuation": " convert sunlight to sand"}, "idx": 0, "task_name": "openbookqa", "doc_id": 203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis means plants are unable to\nAnswer:", "continuation": " provide food sources for others"}, "idx": 1, "task_name": "openbookqa", "doc_id": 203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis means plants are unable to\nAnswer:", "continuation": " be producers in an ecosystem"}, "idx": 2, "task_name": "openbookqa", "doc_id": 203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis means plants are unable to\nAnswer:", "continuation": " make their own food"}, "idx": 3, "task_name": "openbookqa", "doc_id": 203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Through DNA, a rabbit will have long ears if\nAnswer:", "continuation": " rabbits are born with ears"}, "idx": 0, "task_name": "openbookqa", "doc_id": 204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Through DNA, a rabbit will have long ears if\nAnswer:", "continuation": " there was a lot of food"}, "idx": 1, "task_name": "openbookqa", "doc_id": 204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Through DNA, a rabbit will have long ears if\nAnswer:", "continuation": " genetic contributors had long ears"}, "idx": 2, "task_name": "openbookqa", "doc_id": 204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Through DNA, a rabbit will have long ears if\nAnswer:", "continuation": " parents were also rabbits"}, "idx": 3, "task_name": "openbookqa", "doc_id": 204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are various creatures that live in forests, such as\nAnswer:", "continuation": " giant fish"}, "idx": 0, "task_name": "openbookqa", "doc_id": 205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are various creatures that live in forests, such as\nAnswer:", "continuation": " enormous crabs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are various creatures that live in forests, such as\nAnswer:", "continuation": " whitetails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are various creatures that live in forests, such as\nAnswer:", "continuation": " desert jackals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus protects itself with\nAnswer:", "continuation": " water splashing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus protects itself with\nAnswer:", "continuation": " running fast"}, "idx": 1, "task_name": "openbookqa", "doc_id": 206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus protects itself with\nAnswer:", "continuation": " long hands"}, "idx": 2, "task_name": "openbookqa", "doc_id": 206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An octopus protects itself with\nAnswer:", "continuation": " pigment squirting"}, "idx": 3, "task_name": "openbookqa", "doc_id": 206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Resources decreasing in an environment\nAnswer:", "continuation": " induces organisms to use more of their resources"}, "idx": 0, "task_name": "openbookqa", "doc_id": 207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Resources decreasing in an environment\nAnswer:", "continuation": " causes an increase in use of resources"}, "idx": 1, "task_name": "openbookqa", "doc_id": 207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Resources decreasing in an environment\nAnswer:", "continuation": " causes an uptick in birthrate"}, "idx": 2, "task_name": "openbookqa", "doc_id": 207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Resources decreasing in an environment\nAnswer:", "continuation": " induces organisms to be more economical with resources"}, "idx": 3, "task_name": "openbookqa", "doc_id": 207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "continuation": " insects"}, "idx": 0, "task_name": "openbookqa", "doc_id": 208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "continuation": " weathering"}, "idx": 1, "task_name": "openbookqa", "doc_id": 208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "continuation": " lava"}, "idx": 2, "task_name": "openbookqa", "doc_id": 208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "continuation": " erosion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "continuation": " millions of years"}, "idx": 0, "task_name": "openbookqa", "doc_id": 209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "continuation": " 1 day"}, "idx": 1, "task_name": "openbookqa", "doc_id": 209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "continuation": " 10 days"}, "idx": 2, "task_name": "openbookqa", "doc_id": 209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "continuation": " 100 days"}, "idx": 3, "task_name": "openbookqa", "doc_id": 209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Moon phases\nAnswer:", "continuation": " change the moon into cheese"}, "idx": 0, "task_name": "openbookqa", "doc_id": 210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Moon phases\nAnswer:", "continuation": " alter the way the moon's facade looks"}, "idx": 1, "task_name": "openbookqa", "doc_id": 210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Moon phases\nAnswer:", "continuation": " change moon lakes into vapor"}, "idx": 2, "task_name": "openbookqa", "doc_id": 210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Moon phases\nAnswer:", "continuation": " cause lunar eclipse every day"}, "idx": 3, "task_name": "openbookqa", "doc_id": 210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "continuation": " they would be scalded"}, "idx": 0, "task_name": "openbookqa", "doc_id": 211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "continuation": " organs would remain uneffected"}, "idx": 1, "task_name": "openbookqa", "doc_id": 211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "continuation": " they would begin to decay"}, "idx": 2, "task_name": "openbookqa", "doc_id": 211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "continuation": " they would experience pain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Barnyard bovines\nAnswer:", "continuation": " eat organic chicken"}, "idx": 0, "task_name": "openbookqa", "doc_id": 212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Barnyard bovines\nAnswer:", "continuation": " eat eggs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Barnyard bovines\nAnswer:", "continuation": " eat beef"}, "idx": 2, "task_name": "openbookqa", "doc_id": 212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Barnyard bovines\nAnswer:", "continuation": " eat alfalfa hay"}, "idx": 3, "task_name": "openbookqa", "doc_id": 212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments features\nAnswer:", "continuation": " tropical plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments features\nAnswer:", "continuation": " tons of sun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments features\nAnswer:", "continuation": " massive rain totals"}, "idx": 2, "task_name": "openbookqa", "doc_id": 213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments features\nAnswer:", "continuation": " icy precipitation"}, "idx": 3, "task_name": "openbookqa", "doc_id": 213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "continuation": " army of ants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "continuation": " a community"}, "idx": 1, "task_name": "openbookqa", "doc_id": 214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "continuation": " a toy store"}, "idx": 2, "task_name": "openbookqa", "doc_id": 214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "continuation": " a shopping mall"}, "idx": 3, "task_name": "openbookqa", "doc_id": 214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "continuation": " the lungs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "continuation": " the kidney"}, "idx": 1, "task_name": "openbookqa", "doc_id": 215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "continuation": " the heart"}, "idx": 2, "task_name": "openbookqa", "doc_id": 215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "continuation": " the liver"}, "idx": 3, "task_name": "openbookqa", "doc_id": 215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which pair don't reproduce the same way?\nAnswer:", "continuation": " rabbit and hare"}, "idx": 0, "task_name": "openbookqa", "doc_id": 216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which pair don't reproduce the same way?\nAnswer:", "continuation": " mule and hinny"}, "idx": 1, "task_name": "openbookqa", "doc_id": 216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which pair don't reproduce the same way?\nAnswer:", "continuation": " cat and catfish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which pair don't reproduce the same way?\nAnswer:", "continuation": " caterpillar and butterfly"}, "idx": 3, "task_name": "openbookqa", "doc_id": 216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is warm blooded?\nAnswer:", "continuation": " toad"}, "idx": 0, "task_name": "openbookqa", "doc_id": 217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is warm blooded?\nAnswer:", "continuation": " snake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is warm blooded?\nAnswer:", "continuation": " turtle"}, "idx": 2, "task_name": "openbookqa", "doc_id": 217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is warm blooded?\nAnswer:", "continuation": " skunk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "continuation": " a manager"}, "idx": 0, "task_name": "openbookqa", "doc_id": 218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "continuation": " an agent"}, "idx": 1, "task_name": "openbookqa", "doc_id": 218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "continuation": " a meal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "continuation": " a reproductive companion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Vast quantities of metal can be obtained from\nAnswer:", "continuation": " a quarry"}, "idx": 0, "task_name": "openbookqa", "doc_id": 219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Vast quantities of metal can be obtained from\nAnswer:", "continuation": " concerts"}, "idx": 1, "task_name": "openbookqa", "doc_id": 219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Vast quantities of metal can be obtained from\nAnswer:", "continuation": " forests"}, "idx": 2, "task_name": "openbookqa", "doc_id": 219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Vast quantities of metal can be obtained from\nAnswer:", "continuation": " salt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Roasting a turkey requires adding what type of energy\nAnswer:", "continuation": " Heat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Roasting a turkey requires adding what type of energy\nAnswer:", "continuation": " Kinetic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Roasting a turkey requires adding what type of energy\nAnswer:", "continuation": " Magnetic"}, "idx": 2, "task_name": "openbookqa", "doc_id": 220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Roasting a turkey requires adding what type of energy\nAnswer:", "continuation": " Chemical"}, "idx": 3, "task_name": "openbookqa", "doc_id": 220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dry environments often\nAnswer:", "continuation": " liberally use water for everything"}, "idx": 0, "task_name": "openbookqa", "doc_id": 221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dry environments often\nAnswer:", "continuation": " allow plants to flourish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dry environments often\nAnswer:", "continuation": " require people to move"}, "idx": 2, "task_name": "openbookqa", "doc_id": 221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dry environments often\nAnswer:", "continuation": " institute rules about water usage"}, "idx": 3, "task_name": "openbookqa", "doc_id": 221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Phloem moves things around a plant similar to how\nAnswer:", "continuation": " blood moves in a body"}, "idx": 0, "task_name": "openbookqa", "doc_id": 222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Phloem moves things around a plant similar to how\nAnswer:", "continuation": " leaves move in the wind"}, "idx": 1, "task_name": "openbookqa", "doc_id": 222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Phloem moves things around a plant similar to how\nAnswer:", "continuation": " water moves in a system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Phloem moves things around a plant similar to how\nAnswer:", "continuation": " cars move on a street"}, "idx": 3, "task_name": "openbookqa", "doc_id": 222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a polar bear be most comfortable?\nAnswer:", "continuation": " Arizona"}, "idx": 0, "task_name": "openbookqa", "doc_id": 223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a polar bear be most comfortable?\nAnswer:", "continuation": " Georgia"}, "idx": 1, "task_name": "openbookqa", "doc_id": 223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a polar bear be most comfortable?\nAnswer:", "continuation": " Florida"}, "idx": 2, "task_name": "openbookqa", "doc_id": 223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would a polar bear be most comfortable?\nAnswer:", "continuation": " Nebraska"}, "idx": 3, "task_name": "openbookqa", "doc_id": 223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "continuation": " light is far away"}, "idx": 0, "task_name": "openbookqa", "doc_id": 224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "continuation": " light is more easily seen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "continuation": " light is more distant"}, "idx": 2, "task_name": "openbookqa", "doc_id": 224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "continuation": " light is further away"}, "idx": 3, "task_name": "openbookqa", "doc_id": 224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Members of rock bands often perform with\nAnswer:", "continuation": " flutes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Members of rock bands often perform with\nAnswer:", "continuation": " sandals"}, "idx": 1, "task_name": "openbookqa", "doc_id": 225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Members of rock bands often perform with\nAnswer:", "continuation": " earplugs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Members of rock bands often perform with\nAnswer:", "continuation": " gloves"}, "idx": 3, "task_name": "openbookqa", "doc_id": 225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Muscles move bones to produce movement like when\nAnswer:", "continuation": " arms are resting"}, "idx": 0, "task_name": "openbookqa", "doc_id": 226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Muscles move bones to produce movement like when\nAnswer:", "continuation": " hair is growing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Muscles move bones to produce movement like when\nAnswer:", "continuation": " smiles are invisible"}, "idx": 2, "task_name": "openbookqa", "doc_id": 226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Muscles move bones to produce movement like when\nAnswer:", "continuation": " toes are wiggled"}, "idx": 3, "task_name": "openbookqa", "doc_id": 226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How is electricity produced from the ocean?\nAnswer:", "continuation": " decaying organic material from sealife"}, "idx": 0, "task_name": "openbookqa", "doc_id": 227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How is electricity produced from the ocean?\nAnswer:", "continuation": " energy is accessed underwater from tides"}, "idx": 1, "task_name": "openbookqa", "doc_id": 227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How is electricity produced from the ocean?\nAnswer:", "continuation": " drills to access oil supplies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How is electricity produced from the ocean?\nAnswer:", "continuation": " chemical reactions produced from the salt in the water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: small reptile's diet consists mostly of\nAnswer:", "continuation": " invertebrates"}, "idx": 0, "task_name": "openbookqa", "doc_id": 228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: small reptile's diet consists mostly of\nAnswer:", "continuation": " insects"}, "idx": 1, "task_name": "openbookqa", "doc_id": 228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: small reptile's diet consists mostly of\nAnswer:", "continuation": " mammals"}, "idx": 2, "task_name": "openbookqa", "doc_id": 228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: small reptile's diet consists mostly of\nAnswer:", "continuation": " fish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The life work of a flower is to\nAnswer:", "continuation": " provide nice scents"}, "idx": 0, "task_name": "openbookqa", "doc_id": 229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The life work of a flower is to\nAnswer:", "continuation": " be successfully fertilized"}, "idx": 1, "task_name": "openbookqa", "doc_id": 229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The life work of a flower is to\nAnswer:", "continuation": " grow very tall"}, "idx": 2, "task_name": "openbookqa", "doc_id": 229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The life work of a flower is to\nAnswer:", "continuation": " look pretty"}, "idx": 3, "task_name": "openbookqa", "doc_id": 229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "continuation": " the plastic water bottle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "continuation": " the backside of a person"}, "idx": 1, "task_name": "openbookqa", "doc_id": 230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "continuation": " the hair on a doll"}, "idx": 2, "task_name": "openbookqa", "doc_id": 230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "continuation": " the chassis of a car"}, "idx": 3, "task_name": "openbookqa", "doc_id": 230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: During landslides there is often a lot of\nAnswer:", "continuation": " air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: During landslides there is often a lot of\nAnswer:", "continuation": " mud"}, "idx": 1, "task_name": "openbookqa", "doc_id": 231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: During landslides there is often a lot of\nAnswer:", "continuation": " snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: During landslides there is often a lot of\nAnswer:", "continuation": " wind"}, "idx": 3, "task_name": "openbookqa", "doc_id": 231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of a chemical reaction would be\nAnswer:", "continuation": " A rusty fence"}, "idx": 0, "task_name": "openbookqa", "doc_id": 232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of a chemical reaction would be\nAnswer:", "continuation": " Sleeping"}, "idx": 1, "task_name": "openbookqa", "doc_id": 232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of a chemical reaction would be\nAnswer:", "continuation": " Drinking water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of a chemical reaction would be\nAnswer:", "continuation": " Rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "continuation": " equal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "continuation": " inverse"}, "idx": 1, "task_name": "openbookqa", "doc_id": 233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "continuation": " direct"}, "idx": 2, "task_name": "openbookqa", "doc_id": 233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "continuation": " reverse"}, "idx": 3, "task_name": "openbookqa", "doc_id": 233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tuna primarily eat\nAnswer:", "continuation": " parasites, soybeans and flaxseeds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tuna primarily eat\nAnswer:", "continuation": " sea turtles, sharks and coral reefs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tuna primarily eat\nAnswer:", "continuation": " spineless marine organisms, cartilaginous and gelatinous organisms"}, "idx": 2, "task_name": "openbookqa", "doc_id": 234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tuna primarily eat\nAnswer:", "continuation": " sea vegetables like kelp, Irish moss and Arame"}, "idx": 3, "task_name": "openbookqa", "doc_id": 234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation of water can lead to\nAnswer:", "continuation": " waterfalls"}, "idx": 0, "task_name": "openbookqa", "doc_id": 235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation of water can lead to\nAnswer:", "continuation": " blizzards"}, "idx": 1, "task_name": "openbookqa", "doc_id": 235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation of water can lead to\nAnswer:", "continuation": " earthquakes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation of water can lead to\nAnswer:", "continuation": " hot springs"}, "idx": 3, "task_name": "openbookqa", "doc_id": 235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an electrical energy conductor?\nAnswer:", "continuation": " horseshoe"}, "idx": 0, "task_name": "openbookqa", "doc_id": 236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an electrical energy conductor?\nAnswer:", "continuation": " tire"}, "idx": 1, "task_name": "openbookqa", "doc_id": 236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an electrical energy conductor?\nAnswer:", "continuation": " cotton shirt"}, "idx": 2, "task_name": "openbookqa", "doc_id": 236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an electrical energy conductor?\nAnswer:", "continuation": " maple branch"}, "idx": 3, "task_name": "openbookqa", "doc_id": 236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of substance will cool when it touches a cold object?\nAnswer:", "continuation": " warm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of substance will cool when it touches a cold object?\nAnswer:", "continuation": " frozen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of substance will cool when it touches a cold object?\nAnswer:", "continuation": " chilly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of substance will cool when it touches a cold object?\nAnswer:", "continuation": " cold"}, "idx": 3, "task_name": "openbookqa", "doc_id": 237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Scraping an object\nAnswer:", "continuation": " may cause the object to grow in size"}, "idx": 0, "task_name": "openbookqa", "doc_id": 238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Scraping an object\nAnswer:", "continuation": " may cause the object to fall"}, "idx": 1, "task_name": "openbookqa", "doc_id": 238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Scraping an object\nAnswer:", "continuation": " may cause pieces to flake off the object"}, "idx": 2, "task_name": "openbookqa", "doc_id": 238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Scraping an object\nAnswer:", "continuation": " may cause the object to snap in half"}, "idx": 3, "task_name": "openbookqa", "doc_id": 238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pulley is used to lift a flag on a flagpole by\nAnswer:", "continuation": " moving a rope sideways"}, "idx": 0, "task_name": "openbookqa", "doc_id": 239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pulley is used to lift a flag on a flagpole by\nAnswer:", "continuation": " putting something in the air"}, "idx": 1, "task_name": "openbookqa", "doc_id": 239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pulley is used to lift a flag on a flagpole by\nAnswer:", "continuation": " moving things with wheels"}, "idx": 2, "task_name": "openbookqa", "doc_id": 239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pulley is used to lift a flag on a flagpole by\nAnswer:", "continuation": " yanking string up a wheel"}, "idx": 3, "task_name": "openbookqa", "doc_id": 239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal lays eggs\nAnswer:", "continuation": " emus"}, "idx": 0, "task_name": "openbookqa", "doc_id": 240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal lays eggs\nAnswer:", "continuation": " dogs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal lays eggs\nAnswer:", "continuation": " squirrels"}, "idx": 2, "task_name": "openbookqa", "doc_id": 240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal lays eggs\nAnswer:", "continuation": " giraffes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dunes can be made out of the same thing as\nAnswer:", "continuation": " clothes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dunes can be made out of the same thing as\nAnswer:", "continuation": " food"}, "idx": 1, "task_name": "openbookqa", "doc_id": 241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dunes can be made out of the same thing as\nAnswer:", "continuation": " forests"}, "idx": 2, "task_name": "openbookqa", "doc_id": 241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dunes can be made out of the same thing as\nAnswer:", "continuation": " castles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A learned behavior is exhibited when\nAnswer:", "continuation": " squinting in bright light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A learned behavior is exhibited when\nAnswer:", "continuation": " inhaling and exhaling during sleep"}, "idx": 1, "task_name": "openbookqa", "doc_id": 242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A learned behavior is exhibited when\nAnswer:", "continuation": " blinking and gulping air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A learned behavior is exhibited when\nAnswer:", "continuation": " nailing up a picture frame"}, "idx": 3, "task_name": "openbookqa", "doc_id": 242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wax can be used similarly to\nAnswer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wax can be used similarly to\nAnswer:", "continuation": " rubber"}, "idx": 1, "task_name": "openbookqa", "doc_id": 243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wax can be used similarly to\nAnswer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wax can be used similarly to\nAnswer:", "continuation": " metal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "continuation": " get bigger yields"}, "idx": 0, "task_name": "openbookqa", "doc_id": 244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "continuation": " make things boring"}, "idx": 1, "task_name": "openbookqa", "doc_id": 244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "continuation": " keep things random"}, "idx": 2, "task_name": "openbookqa", "doc_id": 244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "continuation": " get smaller yields"}, "idx": 3, "task_name": "openbookqa", "doc_id": 244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "continuation": " feathers slows heat transfer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "continuation": " the natural duck wax"}, "idx": 1, "task_name": "openbookqa", "doc_id": 245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "continuation": " a synthetic thick liner"}, "idx": 2, "task_name": "openbookqa", "doc_id": 245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "continuation": " small flexible solar panels"}, "idx": 3, "task_name": "openbookqa", "doc_id": 245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Turbines churning seawater can be used to produce what?\nAnswer:", "continuation": " a charge for appliances"}, "idx": 0, "task_name": "openbookqa", "doc_id": 246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Turbines churning seawater can be used to produce what?\nAnswer:", "continuation": " large quantities of soup"}, "idx": 1, "task_name": "openbookqa", "doc_id": 246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Turbines churning seawater can be used to produce what?\nAnswer:", "continuation": " large schools of fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Turbines churning seawater can be used to produce what?\nAnswer:", "continuation": " creating some sharp cheese"}, "idx": 3, "task_name": "openbookqa", "doc_id": 246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will increase when a substance absorbs solar energy?\nAnswer:", "continuation": " weight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will increase when a substance absorbs solar energy?\nAnswer:", "continuation": " height"}, "idx": 1, "task_name": "openbookqa", "doc_id": 247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will increase when a substance absorbs solar energy?\nAnswer:", "continuation": " hotness"}, "idx": 2, "task_name": "openbookqa", "doc_id": 247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will increase when a substance absorbs solar energy?\nAnswer:", "continuation": " nutrition"}, "idx": 3, "task_name": "openbookqa", "doc_id": 247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these travels through the air?\nAnswer:", "continuation": " planets"}, "idx": 0, "task_name": "openbookqa", "doc_id": 248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these travels through the air?\nAnswer:", "continuation": " thoughts"}, "idx": 1, "task_name": "openbookqa", "doc_id": 248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these travels through the air?\nAnswer:", "continuation": " automobile"}, "idx": 2, "task_name": "openbookqa", "doc_id": 248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these travels through the air?\nAnswer:", "continuation": " music"}, "idx": 3, "task_name": "openbookqa", "doc_id": 248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A company makes notebooks for college courses, so their main material is\nAnswer:", "continuation": " chips"}, "idx": 0, "task_name": "openbookqa", "doc_id": 249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A company makes notebooks for college courses, so their main material is\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A company makes notebooks for college courses, so their main material is\nAnswer:", "continuation": " grass"}, "idx": 2, "task_name": "openbookqa", "doc_id": 249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A company makes notebooks for college courses, so their main material is\nAnswer:", "continuation": " trees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rain is usually guaranteed when all are present but\nAnswer:", "continuation": " cirrus clouds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rain is usually guaranteed when all are present but\nAnswer:", "continuation": " cumulus clouds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rain is usually guaranteed when all are present but\nAnswer:", "continuation": " hail stones"}, "idx": 2, "task_name": "openbookqa", "doc_id": 250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Rain is usually guaranteed when all are present but\nAnswer:", "continuation": " direct sunshine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of air pollution are\nAnswer:", "continuation": " Walking"}, "idx": 0, "task_name": "openbookqa", "doc_id": 251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of air pollution are\nAnswer:", "continuation": " Landfills"}, "idx": 1, "task_name": "openbookqa", "doc_id": 251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of air pollution are\nAnswer:", "continuation": " Water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Sources of air pollution are\nAnswer:", "continuation": " Chips"}, "idx": 3, "task_name": "openbookqa", "doc_id": 251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are iron nails made out of?\nAnswer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are iron nails made out of?\nAnswer:", "continuation": " plastic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are iron nails made out of?\nAnswer:", "continuation": " metal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are iron nails made out of?\nAnswer:", "continuation": " glass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "continuation": " combining"}, "idx": 0, "task_name": "openbookqa", "doc_id": 253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "continuation": " drinking"}, "idx": 1, "task_name": "openbookqa", "doc_id": 253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "continuation": " Subtracting"}, "idx": 2, "task_name": "openbookqa", "doc_id": 253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "continuation": " throwing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A moth leaving it's cocoon is the final step in a\nAnswer:", "continuation": " life cycle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A moth leaving it's cocoon is the final step in a\nAnswer:", "continuation": " transformation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A moth leaving it's cocoon is the final step in a\nAnswer:", "continuation": " recycling process"}, "idx": 2, "task_name": "openbookqa", "doc_id": 254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A moth leaving it's cocoon is the final step in a\nAnswer:", "continuation": " chemical reaction"}, "idx": 3, "task_name": "openbookqa", "doc_id": 254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "continuation": " ice and snow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "continuation": " sticks and stones"}, "idx": 1, "task_name": "openbookqa", "doc_id": 255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "continuation": " snow and water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "continuation": " iron and carbon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is powered the same way an electric car is?\nAnswer:", "continuation": " a bicycle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is powered the same way an electric car is?\nAnswer:", "continuation": " a motorcycle"}, "idx": 1, "task_name": "openbookqa", "doc_id": 256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is powered the same way an electric car is?\nAnswer:", "continuation": " a propane grill"}, "idx": 2, "task_name": "openbookqa", "doc_id": 256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is powered the same way an electric car is?\nAnswer:", "continuation": " a blender"}, "idx": 3, "task_name": "openbookqa", "doc_id": 256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Aluminum is what?\nAnswer:", "continuation": " reprocessable"}, "idx": 0, "task_name": "openbookqa", "doc_id": 257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Aluminum is what?\nAnswer:", "continuation": " plastic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Aluminum is what?\nAnswer:", "continuation": " liquid"}, "idx": 2, "task_name": "openbookqa", "doc_id": 257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Aluminum is what?\nAnswer:", "continuation": " absorbent"}, "idx": 3, "task_name": "openbookqa", "doc_id": 257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To get warm frogs can\nAnswer:", "continuation": " wear a Christmas sweater"}, "idx": 0, "task_name": "openbookqa", "doc_id": 258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To get warm frogs can\nAnswer:", "continuation": " Drink a hot chocolate"}, "idx": 1, "task_name": "openbookqa", "doc_id": 258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To get warm frogs can\nAnswer:", "continuation": " Go for a run"}, "idx": 2, "task_name": "openbookqa", "doc_id": 258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To get warm frogs can\nAnswer:", "continuation": " sit under a lamp"}, "idx": 3, "task_name": "openbookqa", "doc_id": 258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "continuation": " it exhibits absolute rigidity"}, "idx": 0, "task_name": "openbookqa", "doc_id": 259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "continuation": " it is a solid mass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "continuation": " it is a variable substance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthworms create tunnels in\nAnswer:", "continuation": " ice"}, "idx": 0, "task_name": "openbookqa", "doc_id": 260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthworms create tunnels in\nAnswer:", "continuation": " dirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthworms create tunnels in\nAnswer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earthworms create tunnels in\nAnswer:", "continuation": " concrete"}, "idx": 3, "task_name": "openbookqa", "doc_id": 260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "continuation": " regurgitation"}, "idx": 0, "task_name": "openbookqa", "doc_id": 261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "continuation": " decomposition"}, "idx": 1, "task_name": "openbookqa", "doc_id": 261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "continuation": " recycling"}, "idx": 2, "task_name": "openbookqa", "doc_id": 261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "continuation": " burial"}, "idx": 3, "task_name": "openbookqa", "doc_id": 261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The aluminum cans were much hotter than the\nAnswer:", "continuation": " gold jewelry"}, "idx": 0, "task_name": "openbookqa", "doc_id": 262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The aluminum cans were much hotter than the\nAnswer:", "continuation": " wooden fence"}, "idx": 1, "task_name": "openbookqa", "doc_id": 262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The aluminum cans were much hotter than the\nAnswer:", "continuation": " brass doorknob"}, "idx": 2, "task_name": "openbookqa", "doc_id": 262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The aluminum cans were much hotter than the\nAnswer:", "continuation": " steel pole"}, "idx": 3, "task_name": "openbookqa", "doc_id": 262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is not an input in photosynthesis?\nAnswer:", "continuation": " sunlight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is not an input in photosynthesis?\nAnswer:", "continuation": " oxygen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is not an input in photosynthesis?\nAnswer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following is not an input in photosynthesis?\nAnswer:", "continuation": " carbon dioxide"}, "idx": 3, "task_name": "openbookqa", "doc_id": 263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these are you most likely to find in a desert?\nAnswer:", "continuation": " a hammer head shark"}, "idx": 0, "task_name": "openbookqa", "doc_id": 264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these are you most likely to find in a desert?\nAnswer:", "continuation": " a big tilapia fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these are you most likely to find in a desert?\nAnswer:", "continuation": " a prickly horned male lizard"}, "idx": 2, "task_name": "openbookqa", "doc_id": 264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these are you most likely to find in a desert?\nAnswer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "continuation": " it will be fine"}, "idx": 0, "task_name": "openbookqa", "doc_id": 265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "continuation": " it will be happy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "continuation": " it will be comfortable"}, "idx": 2, "task_name": "openbookqa", "doc_id": 265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "continuation": " It will pass out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following can be used to turn on an electrical device?\nAnswer:", "continuation": " solar-rechargeable battery"}, "idx": 0, "task_name": "openbookqa", "doc_id": 266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following can be used to turn on an electrical device?\nAnswer:", "continuation": " a wedge"}, "idx": 1, "task_name": "openbookqa", "doc_id": 266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following can be used to turn on an electrical device?\nAnswer:", "continuation": " a magnet"}, "idx": 2, "task_name": "openbookqa", "doc_id": 266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following can be used to turn on an electrical device?\nAnswer:", "continuation": " pressure gauge"}, "idx": 3, "task_name": "openbookqa", "doc_id": 266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following contains large amounts of salt water?\nAnswer:", "continuation": " The Amazon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following contains large amounts of salt water?\nAnswer:", "continuation": " The Nile"}, "idx": 1, "task_name": "openbookqa", "doc_id": 267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following contains large amounts of salt water?\nAnswer:", "continuation": " The Indian"}, "idx": 2, "task_name": "openbookqa", "doc_id": 267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following contains large amounts of salt water?\nAnswer:", "continuation": " The Mississippi"}, "idx": 3, "task_name": "openbookqa", "doc_id": 267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "continuation": " the butterfly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "continuation": " the sloth"}, "idx": 1, "task_name": "openbookqa", "doc_id": 268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "continuation": " the praying mantis"}, "idx": 2, "task_name": "openbookqa", "doc_id": 268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "continuation": " the antelope"}, "idx": 3, "task_name": "openbookqa", "doc_id": 268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person speaks English as her first language because\nAnswer:", "continuation": " media is mainly in English"}, "idx": 0, "task_name": "openbookqa", "doc_id": 269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person speaks English as her first language because\nAnswer:", "continuation": " school is in English"}, "idx": 1, "task_name": "openbookqa", "doc_id": 269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person speaks English as her first language because\nAnswer:", "continuation": " she was genetically predisposed"}, "idx": 2, "task_name": "openbookqa", "doc_id": 269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person speaks English as her first language because\nAnswer:", "continuation": " she watched her parents speak"}, "idx": 3, "task_name": "openbookqa", "doc_id": 269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what are eaten by honey producing insects?\nAnswer:", "continuation": " they consume plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what are eaten by honey producing insects?\nAnswer:", "continuation": " they eat cows"}, "idx": 1, "task_name": "openbookqa", "doc_id": 270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what are eaten by honey producing insects?\nAnswer:", "continuation": " plant reproduction parts"}, "idx": 2, "task_name": "openbookqa", "doc_id": 270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what are eaten by honey producing insects?\nAnswer:", "continuation": " they eat flowers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Earth's closest heat source is\nAnswer:", "continuation": " our celestial fireball"}, "idx": 0, "task_name": "openbookqa", "doc_id": 271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Earth's closest heat source is\nAnswer:", "continuation": " solar flares"}, "idx": 1, "task_name": "openbookqa", "doc_id": 271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Earth's closest heat source is\nAnswer:", "continuation": " gamma rays"}, "idx": 2, "task_name": "openbookqa", "doc_id": 271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Earth's closest heat source is\nAnswer:", "continuation": " big bang"}, "idx": 3, "task_name": "openbookqa", "doc_id": 271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see how wind effects sand?\nAnswer:", "continuation": " sand is always moving"}, "idx": 0, "task_name": "openbookqa", "doc_id": 272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see how wind effects sand?\nAnswer:", "continuation": " sandstorms create ripples in sand"}, "idx": 1, "task_name": "openbookqa", "doc_id": 272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see how wind effects sand?\nAnswer:", "continuation": " sand is easy to move through"}, "idx": 2, "task_name": "openbookqa", "doc_id": 272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How can we see how wind effects sand?\nAnswer:", "continuation": " beaches often have waves in the sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fruit comes from what source\nAnswer:", "continuation": " an organism that releases carbon dioxide"}, "idx": 0, "task_name": "openbookqa", "doc_id": 273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fruit comes from what source\nAnswer:", "continuation": " an organism that absorbs water through it's branches"}, "idx": 1, "task_name": "openbookqa", "doc_id": 273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fruit comes from what source\nAnswer:", "continuation": " an organism that absorbs oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fruit comes from what source\nAnswer:", "continuation": " an organism that absorbs water through it's roots"}, "idx": 3, "task_name": "openbookqa", "doc_id": 273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In which location would a groundhog hide from a wolf?\nAnswer:", "continuation": " beside a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In which location would a groundhog hide from a wolf?\nAnswer:", "continuation": " in the grass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In which location would a groundhog hide from a wolf?\nAnswer:", "continuation": " on a stump"}, "idx": 2, "task_name": "openbookqa", "doc_id": 274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In which location would a groundhog hide from a wolf?\nAnswer:", "continuation": " under the ground"}, "idx": 3, "task_name": "openbookqa", "doc_id": 274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cutting down trees in a forest\nAnswer:", "continuation": " leads to more habitats for animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cutting down trees in a forest\nAnswer:", "continuation": " decreases the chance of erosion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cutting down trees in a forest\nAnswer:", "continuation": " increases the number of trees in the forest"}, "idx": 2, "task_name": "openbookqa", "doc_id": 275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cutting down trees in a forest\nAnswer:", "continuation": " leads to less habitats for animals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "continuation": " animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "continuation": " resistance"}, "idx": 1, "task_name": "openbookqa", "doc_id": 276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "continuation": " sunshine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The arctic is white in coloring\nAnswer:", "continuation": " because it's overpopulated with polar bears"}, "idx": 0, "task_name": "openbookqa", "doc_id": 277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The arctic is white in coloring\nAnswer:", "continuation": " because it's covered in white lilies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The arctic is white in coloring\nAnswer:", "continuation": " because it's blanketed in crystalline ice water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The arctic is white in coloring\nAnswer:", "continuation": " because it's gets so little sunlight"}, "idx": 3, "task_name": "openbookqa", "doc_id": 277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "continuation": " Lots of meat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "continuation": " Lots of love"}, "idx": 1, "task_name": "openbookqa", "doc_id": 278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "continuation": " Vitamin supplements"}, "idx": 2, "task_name": "openbookqa", "doc_id": 278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "continuation": " An insulated room"}, "idx": 3, "task_name": "openbookqa", "doc_id": 278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The majority of a lizard's diet consists of\nAnswer:", "continuation": " fleas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The majority of a lizard's diet consists of\nAnswer:", "continuation": " crawlies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The majority of a lizard's diet consists of\nAnswer:", "continuation": " gummy worms"}, "idx": 2, "task_name": "openbookqa", "doc_id": 279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The majority of a lizard's diet consists of\nAnswer:", "continuation": " berries"}, "idx": 3, "task_name": "openbookqa", "doc_id": 279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What food production happens in a leaf?\nAnswer:", "continuation": " nutrient making process"}, "idx": 0, "task_name": "openbookqa", "doc_id": 280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What food production happens in a leaf?\nAnswer:", "continuation": " the breathing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What food production happens in a leaf?\nAnswer:", "continuation": " the respiration"}, "idx": 2, "task_name": "openbookqa", "doc_id": 280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What food production happens in a leaf?\nAnswer:", "continuation": " the digestion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plants are unable to grow if they have zero access to\nAnswer:", "continuation": " a nice cool breeze"}, "idx": 0, "task_name": "openbookqa", "doc_id": 281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plants are unable to grow if they have zero access to\nAnswer:", "continuation": " fresh soil with manure"}, "idx": 1, "task_name": "openbookqa", "doc_id": 281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plants are unable to grow if they have zero access to\nAnswer:", "continuation": " a regular source of saltwater"}, "idx": 2, "task_name": "openbookqa", "doc_id": 281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Plants are unable to grow if they have zero access to\nAnswer:", "continuation": " needs required for creating chlorophyll"}, "idx": 3, "task_name": "openbookqa", "doc_id": 281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which characteristic did a person inherit?\nAnswer:", "continuation": " length of hair"}, "idx": 0, "task_name": "openbookqa", "doc_id": 282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which characteristic did a person inherit?\nAnswer:", "continuation": " number of friends"}, "idx": 1, "task_name": "openbookqa", "doc_id": 282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which characteristic did a person inherit?\nAnswer:", "continuation": " number of nails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which characteristic did a person inherit?\nAnswer:", "continuation": " length of shirt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Selective deforestation has a negative impact on\nAnswer:", "continuation": " rain clouds and ozone layer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Selective deforestation has a negative impact on\nAnswer:", "continuation": " lakes, ponds and shellfish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Selective deforestation has a negative impact on\nAnswer:", "continuation": " greenhouse gases and algae"}, "idx": 2, "task_name": "openbookqa", "doc_id": 283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Selective deforestation has a negative impact on\nAnswer:", "continuation": " living organisms in ecosystem"}, "idx": 3, "task_name": "openbookqa", "doc_id": 283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would you find a mine?\nAnswer:", "continuation": " in a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would you find a mine?\nAnswer:", "continuation": " under a mountain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would you find a mine?\nAnswer:", "continuation": " in the air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Where would you find a mine?\nAnswer:", "continuation": " in the water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can cause people to crash their car?\nAnswer:", "continuation": " Seeing a solar eclipse"}, "idx": 0, "task_name": "openbookqa", "doc_id": 285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can cause people to crash their car?\nAnswer:", "continuation": " Using their turn signals"}, "idx": 1, "task_name": "openbookqa", "doc_id": 285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can cause people to crash their car?\nAnswer:", "continuation": " Driving the speed limit"}, "idx": 2, "task_name": "openbookqa", "doc_id": 285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What can cause people to crash their car?\nAnswer:", "continuation": " Keeping their eyes on the road"}, "idx": 3, "task_name": "openbookqa", "doc_id": 285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coral grows in\nAnswer:", "continuation": " frigid waters"}, "idx": 0, "task_name": "openbookqa", "doc_id": 286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coral grows in\nAnswer:", "continuation": " tepid seas"}, "idx": 1, "task_name": "openbookqa", "doc_id": 286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coral grows in\nAnswer:", "continuation": " glacial environments"}, "idx": 2, "task_name": "openbookqa", "doc_id": 286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coral grows in\nAnswer:", "continuation": " jungle forests"}, "idx": 3, "task_name": "openbookqa", "doc_id": 286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "continuation": " certain things may be featured"}, "idx": 0, "task_name": "openbookqa", "doc_id": 287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "continuation": " certain features may be predicted"}, "idx": 1, "task_name": "openbookqa", "doc_id": 287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "continuation": " certain traits may be given"}, "idx": 2, "task_name": "openbookqa", "doc_id": 287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "continuation": " certain features may be guaranteed"}, "idx": 3, "task_name": "openbookqa", "doc_id": 287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "continuation": " longer antlers"}, "idx": 0, "task_name": "openbookqa", "doc_id": 288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "continuation": " food for itself"}, "idx": 1, "task_name": "openbookqa", "doc_id": 288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "continuation": " baby deer"}, "idx": 2, "task_name": "openbookqa", "doc_id": 288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "continuation": " urine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Building new areas to dispose of refuse may lead to\nAnswer:", "continuation": " community concerns over environmental impact"}, "idx": 0, "task_name": "openbookqa", "doc_id": 289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Building new areas to dispose of refuse may lead to\nAnswer:", "continuation": " better air and soil quality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Building new areas to dispose of refuse may lead to\nAnswer:", "continuation": " higher value on land parcels"}, "idx": 2, "task_name": "openbookqa", "doc_id": 289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Building new areas to dispose of refuse may lead to\nAnswer:", "continuation": " improvement in water supply"}, "idx": 3, "task_name": "openbookqa", "doc_id": 289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " causes puddles to become dried out mud"}, "idx": 0, "task_name": "openbookqa", "doc_id": 290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " causes fields of crops to grow faster"}, "idx": 1, "task_name": "openbookqa", "doc_id": 290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " causes flowers to bloom abundantly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Evaporation\nAnswer:", "continuation": " fills up irrigation ponds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "continuation": " pythons"}, "idx": 0, "task_name": "openbookqa", "doc_id": 291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "continuation": " salmon"}, "idx": 1, "task_name": "openbookqa", "doc_id": 291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "continuation": " robins"}, "idx": 2, "task_name": "openbookqa", "doc_id": 291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "continuation": " craters"}, "idx": 3, "task_name": "openbookqa", "doc_id": 291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item urinates?\nAnswer:", "continuation": " airplane"}, "idx": 0, "task_name": "openbookqa", "doc_id": 292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item urinates?\nAnswer:", "continuation": " car"}, "idx": 1, "task_name": "openbookqa", "doc_id": 292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item urinates?\nAnswer:", "continuation": " mammal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item urinates?\nAnswer:", "continuation": " boat"}, "idx": 3, "task_name": "openbookqa", "doc_id": 292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: All of the following are examples of evaporation apart from\nAnswer:", "continuation": " Warm breath fogging up a mirror"}, "idx": 0, "task_name": "openbookqa", "doc_id": 293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: All of the following are examples of evaporation apart from\nAnswer:", "continuation": " Morning dew drying on the grass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: All of the following are examples of evaporation apart from\nAnswer:", "continuation": " The water level in a glass decreasing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: All of the following are examples of evaporation apart from\nAnswer:", "continuation": " Sweat drying on skin"}, "idx": 3, "task_name": "openbookqa", "doc_id": 293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lunar cycle also changes\nAnswer:", "continuation": " water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lunar cycle also changes\nAnswer:", "continuation": " colors"}, "idx": 1, "task_name": "openbookqa", "doc_id": 294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lunar cycle also changes\nAnswer:", "continuation": " the sun"}, "idx": 2, "task_name": "openbookqa", "doc_id": 294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The lunar cycle also changes\nAnswer:", "continuation": " planets"}, "idx": 3, "task_name": "openbookqa", "doc_id": 294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An organism that makes food for itself\nAnswer:", "continuation": " is nutritionally self sustaining"}, "idx": 0, "task_name": "openbookqa", "doc_id": 295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An organism that makes food for itself\nAnswer:", "continuation": " will die faster than other organisms"}, "idx": 1, "task_name": "openbookqa", "doc_id": 295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An organism that makes food for itself\nAnswer:", "continuation": " will need help sustaining strength"}, "idx": 2, "task_name": "openbookqa", "doc_id": 295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An organism that makes food for itself\nAnswer:", "continuation": " is reliant on other organisms for assistance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does the respiratory system transfer to the circulatory system?\nAnswer:", "continuation": " food"}, "idx": 0, "task_name": "openbookqa", "doc_id": 296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does the respiratory system transfer to the circulatory system?\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does the respiratory system transfer to the circulatory system?\nAnswer:", "continuation": " nutrients"}, "idx": 2, "task_name": "openbookqa", "doc_id": 296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What does the respiratory system transfer to the circulatory system?\nAnswer:", "continuation": " O"}, "idx": 3, "task_name": "openbookqa", "doc_id": 296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a closed circuit, electricity will\nAnswer:", "continuation": " burn out"}, "idx": 0, "task_name": "openbookqa", "doc_id": 297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a closed circuit, electricity will\nAnswer:", "continuation": " charge itself"}, "idx": 1, "task_name": "openbookqa", "doc_id": 297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a closed circuit, electricity will\nAnswer:", "continuation": " loop endlessly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a closed circuit, electricity will\nAnswer:", "continuation": " resist flow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "continuation": " mother"}, "idx": 0, "task_name": "openbookqa", "doc_id": 298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "continuation": " grandfather"}, "idx": 1, "task_name": "openbookqa", "doc_id": 298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "continuation": " daughter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "continuation": " father"}, "idx": 3, "task_name": "openbookqa", "doc_id": 298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "continuation": " richer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "continuation": " dryer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "continuation": " sandy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "continuation": " harder"}, "idx": 3, "task_name": "openbookqa", "doc_id": 299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what kind of temperature causes fur shedding?\nAnswer:", "continuation": " in freezing cold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what kind of temperature causes fur shedding?\nAnswer:", "continuation": " a high temperature"}, "idx": 1, "task_name": "openbookqa", "doc_id": 300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what kind of temperature causes fur shedding?\nAnswer:", "continuation": " in any temperature"}, "idx": 2, "task_name": "openbookqa", "doc_id": 300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what kind of temperature causes fur shedding?\nAnswer:", "continuation": " a low temperature"}, "idx": 3, "task_name": "openbookqa", "doc_id": 300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: exposure to fire could result in\nAnswer:", "continuation": " wet items"}, "idx": 0, "task_name": "openbookqa", "doc_id": 301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: exposure to fire could result in\nAnswer:", "continuation": " cold items"}, "idx": 1, "task_name": "openbookqa", "doc_id": 301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: exposure to fire could result in\nAnswer:", "continuation": " none of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: exposure to fire could result in\nAnswer:", "continuation": " combusted items"}, "idx": 3, "task_name": "openbookqa", "doc_id": 301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of clear weather meaning sunny weather?\nAnswer:", "continuation": " more stars are visible on clear nights"}, "idx": 0, "task_name": "openbookqa", "doc_id": 302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of clear weather meaning sunny weather?\nAnswer:", "continuation": " cloud cover protects from sunburn"}, "idx": 1, "task_name": "openbookqa", "doc_id": 302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of clear weather meaning sunny weather?\nAnswer:", "continuation": " clear days will be warmer"}, "idx": 2, "task_name": "openbookqa", "doc_id": 302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of clear weather meaning sunny weather?\nAnswer:", "continuation": " fewer clouds allow for more sunlight"}, "idx": 3, "task_name": "openbookqa", "doc_id": 302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "continuation": " a wick"}, "idx": 0, "task_name": "openbookqa", "doc_id": 303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "continuation": " a funnel"}, "idx": 1, "task_name": "openbookqa", "doc_id": 303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "continuation": " a knife"}, "idx": 2, "task_name": "openbookqa", "doc_id": 303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "continuation": " a whisk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a compass is a kind of tool for determining direction by pointing\nAnswer:", "continuation": " to western Canada shoreline"}, "idx": 0, "task_name": "openbookqa", "doc_id": 304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a compass is a kind of tool for determining direction by pointing\nAnswer:", "continuation": " to the lower pole"}, "idx": 1, "task_name": "openbookqa", "doc_id": 304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a compass is a kind of tool for determining direction by pointing\nAnswer:", "continuation": " to the upper pole"}, "idx": 2, "task_name": "openbookqa", "doc_id": 304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a compass is a kind of tool for determining direction by pointing\nAnswer:", "continuation": " directly to the equator"}, "idx": 3, "task_name": "openbookqa", "doc_id": 304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these saws will last longer?\nAnswer:", "continuation": " iron saw"}, "idx": 0, "task_name": "openbookqa", "doc_id": 305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these saws will last longer?\nAnswer:", "continuation": " aluminium saw"}, "idx": 1, "task_name": "openbookqa", "doc_id": 305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these saws will last longer?\nAnswer:", "continuation": " plastic saw"}, "idx": 2, "task_name": "openbookqa", "doc_id": 305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these saws will last longer?\nAnswer:", "continuation": " wooden saw"}, "idx": 3, "task_name": "openbookqa", "doc_id": 305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "continuation": " the single moon close to us"}, "idx": 0, "task_name": "openbookqa", "doc_id": 306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "continuation": " the orion star cluster"}, "idx": 1, "task_name": "openbookqa", "doc_id": 306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "continuation": " the sun that shines all day"}, "idx": 2, "task_name": "openbookqa", "doc_id": 306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "continuation": " all of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The moon is known for having what feature?\nAnswer:", "continuation": " frozen streams of water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The moon is known for having what feature?\nAnswer:", "continuation": " large bowl shaped cavities"}, "idx": 1, "task_name": "openbookqa", "doc_id": 307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The moon is known for having what feature?\nAnswer:", "continuation": " caves formed by solar winds"}, "idx": 2, "task_name": "openbookqa", "doc_id": 307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The moon is known for having what feature?\nAnswer:", "continuation": " groups of large trees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cellular respiration's trash is\nAnswer:", "continuation": " a bug's treasure"}, "idx": 0, "task_name": "openbookqa", "doc_id": 308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cellular respiration's trash is\nAnswer:", "continuation": " a cow's treasure"}, "idx": 1, "task_name": "openbookqa", "doc_id": 308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cellular respiration's trash is\nAnswer:", "continuation": " a plant's treasure"}, "idx": 2, "task_name": "openbookqa", "doc_id": 308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cellular respiration's trash is\nAnswer:", "continuation": " everyone's trash"}, "idx": 3, "task_name": "openbookqa", "doc_id": 308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "continuation": " swimming in a lake"}, "idx": 0, "task_name": "openbookqa", "doc_id": 309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "continuation": " building a new subdivision"}, "idx": 1, "task_name": "openbookqa", "doc_id": 309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "continuation": " dancing in a field"}, "idx": 2, "task_name": "openbookqa", "doc_id": 309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "continuation": " going for a hike"}, "idx": 3, "task_name": "openbookqa", "doc_id": 309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bird that takes off flying is\nAnswer:", "continuation": " using heat to produce motion"}, "idx": 0, "task_name": "openbookqa", "doc_id": 310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bird that takes off flying is\nAnswer:", "continuation": " using calories to produce motion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bird that takes off flying is\nAnswer:", "continuation": " using wings to produce heat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bird that takes off flying is\nAnswer:", "continuation": " using calories to produce energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The leading cause of soil and rock erosion is\nAnswer:", "continuation": " H2O"}, "idx": 0, "task_name": "openbookqa", "doc_id": 311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The leading cause of soil and rock erosion is\nAnswer:", "continuation": " CO2"}, "idx": 1, "task_name": "openbookqa", "doc_id": 311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The leading cause of soil and rock erosion is\nAnswer:", "continuation": " NaCl"}, "idx": 2, "task_name": "openbookqa", "doc_id": 311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The leading cause of soil and rock erosion is\nAnswer:", "continuation": " Fe"}, "idx": 3, "task_name": "openbookqa", "doc_id": 311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these foods might have a negative impact on humans?\nAnswer:", "continuation": " Organic corn"}, "idx": 0, "task_name": "openbookqa", "doc_id": 312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these foods might have a negative impact on humans?\nAnswer:", "continuation": " Conventional corn"}, "idx": 1, "task_name": "openbookqa", "doc_id": 312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these foods might have a negative impact on humans?\nAnswer:", "continuation": " Organic potato"}, "idx": 2, "task_name": "openbookqa", "doc_id": 312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these foods might have a negative impact on humans?\nAnswer:", "continuation": " Organic Apples"}, "idx": 3, "task_name": "openbookqa", "doc_id": 312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of implement is a compass?\nAnswer:", "continuation": " to test heat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of implement is a compass?\nAnswer:", "continuation": " for wind speed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of implement is a compass?\nAnswer:", "continuation": " it measures distance"}, "idx": 2, "task_name": "openbookqa", "doc_id": 313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of implement is a compass?\nAnswer:", "continuation": " it shows direction"}, "idx": 3, "task_name": "openbookqa", "doc_id": 313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nectar is taken to\nAnswer:", "continuation": " flowers"}, "idx": 0, "task_name": "openbookqa", "doc_id": 314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nectar is taken to\nAnswer:", "continuation": " a hive"}, "idx": 1, "task_name": "openbookqa", "doc_id": 314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nectar is taken to\nAnswer:", "continuation": " a stream"}, "idx": 2, "task_name": "openbookqa", "doc_id": 314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Nectar is taken to\nAnswer:", "continuation": " a nest"}, "idx": 3, "task_name": "openbookqa", "doc_id": 314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what is the closest source of plasma to our planet?\nAnswer:", "continuation": " all of these"}, "idx": 0, "task_name": "openbookqa", "doc_id": 315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what is the closest source of plasma to our planet?\nAnswer:", "continuation": " the cloud in the sky"}, "idx": 1, "task_name": "openbookqa", "doc_id": 315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what is the closest source of plasma to our planet?\nAnswer:", "continuation": " the nearest star sulfur burning heavenly body"}, "idx": 2, "task_name": "openbookqa", "doc_id": 315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what is the closest source of plasma to our planet?\nAnswer:", "continuation": " the bare moon surface"}, "idx": 3, "task_name": "openbookqa", "doc_id": 315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sailor needs to navigate to the shore, and does this by\nAnswer:", "continuation": " closing the sails quickly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sailor needs to navigate to the shore, and does this by\nAnswer:", "continuation": " setting out to sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sailor needs to navigate to the shore, and does this by\nAnswer:", "continuation": " making an adjustment to the rudder"}, "idx": 2, "task_name": "openbookqa", "doc_id": 316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sailor needs to navigate to the shore, and does this by\nAnswer:", "continuation": " taking the afternoon off"}, "idx": 3, "task_name": "openbookqa", "doc_id": 316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat starts its life similarly to a\nAnswer:", "continuation": " chicken"}, "idx": 0, "task_name": "openbookqa", "doc_id": 317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat starts its life similarly to a\nAnswer:", "continuation": " pig"}, "idx": 1, "task_name": "openbookqa", "doc_id": 317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat starts its life similarly to a\nAnswer:", "continuation": " butterfly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bat starts its life similarly to a\nAnswer:", "continuation": " duck"}, "idx": 3, "task_name": "openbookqa", "doc_id": 317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order to catch a rabbit, a predator must be\nAnswer:", "continuation": " big"}, "idx": 0, "task_name": "openbookqa", "doc_id": 318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order to catch a rabbit, a predator must be\nAnswer:", "continuation": " quick"}, "idx": 1, "task_name": "openbookqa", "doc_id": 318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order to catch a rabbit, a predator must be\nAnswer:", "continuation": " slow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order to catch a rabbit, a predator must be\nAnswer:", "continuation": " small"}, "idx": 3, "task_name": "openbookqa", "doc_id": 318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "continuation": " prey"}, "idx": 0, "task_name": "openbookqa", "doc_id": 319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "continuation": " predator"}, "idx": 1, "task_name": "openbookqa", "doc_id": 319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "continuation": " herbivore"}, "idx": 2, "task_name": "openbookqa", "doc_id": 319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "continuation": " canary"}, "idx": 3, "task_name": "openbookqa", "doc_id": 319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A warm-weather organism can be found in\nAnswer:", "continuation": " the Sahara"}, "idx": 0, "task_name": "openbookqa", "doc_id": 320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A warm-weather organism can be found in\nAnswer:", "continuation": " the mountains"}, "idx": 1, "task_name": "openbookqa", "doc_id": 320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A warm-weather organism can be found in\nAnswer:", "continuation": " the ocean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A warm-weather organism can be found in\nAnswer:", "continuation": " the sewers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When approaching an elephant from a great distance,\nAnswer:", "continuation": " it stays large"}, "idx": 0, "task_name": "openbookqa", "doc_id": 321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When approaching an elephant from a great distance,\nAnswer:", "continuation": " it grows larger"}, "idx": 1, "task_name": "openbookqa", "doc_id": 321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When approaching an elephant from a great distance,\nAnswer:", "continuation": " it gets bigger"}, "idx": 2, "task_name": "openbookqa", "doc_id": 321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When approaching an elephant from a great distance,\nAnswer:", "continuation": " it looks bigger"}, "idx": 3, "task_name": "openbookqa", "doc_id": 321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would cause a human to grow?\nAnswer:", "continuation": " light waves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would cause a human to grow?\nAnswer:", "continuation": " eating wheat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would cause a human to grow?\nAnswer:", "continuation": " photosynthesis"}, "idx": 2, "task_name": "openbookqa", "doc_id": 322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would cause a human to grow?\nAnswer:", "continuation": " marching"}, "idx": 3, "task_name": "openbookqa", "doc_id": 322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A saguaro has adaptations for an environment with\nAnswer:", "continuation": " lots of snow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A saguaro has adaptations for an environment with\nAnswer:", "continuation": " many people"}, "idx": 1, "task_name": "openbookqa", "doc_id": 323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A saguaro has adaptations for an environment with\nAnswer:", "continuation": " less water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A saguaro has adaptations for an environment with\nAnswer:", "continuation": " more water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are less hummingbirds by this house than before because of\nAnswer:", "continuation": " a feeder at this house"}, "idx": 0, "task_name": "openbookqa", "doc_id": 324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are less hummingbirds by this house than before because of\nAnswer:", "continuation": " the birds no longer like feeders"}, "idx": 1, "task_name": "openbookqa", "doc_id": 324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are less hummingbirds by this house than before because of\nAnswer:", "continuation": " the size of the feeder"}, "idx": 2, "task_name": "openbookqa", "doc_id": 324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There are less hummingbirds by this house than before because of\nAnswer:", "continuation": " a feeder at another house"}, "idx": 3, "task_name": "openbookqa", "doc_id": 324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the  oceans are full of\nAnswer:", "continuation": " water lilies"}, "idx": 0, "task_name": "openbookqa", "doc_id": 325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the  oceans are full of\nAnswer:", "continuation": " guppies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the  oceans are full of\nAnswer:", "continuation": " sea life"}, "idx": 2, "task_name": "openbookqa", "doc_id": 325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the  oceans are full of\nAnswer:", "continuation": " fresh water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A light was off because the cord was\nAnswer:", "continuation": " sitting on the table"}, "idx": 0, "task_name": "openbookqa", "doc_id": 326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A light was off because the cord was\nAnswer:", "continuation": " attached to the wall"}, "idx": 1, "task_name": "openbookqa", "doc_id": 326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A light was off because the cord was\nAnswer:", "continuation": " attached to an extension cord"}, "idx": 2, "task_name": "openbookqa", "doc_id": 326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A light was off because the cord was\nAnswer:", "continuation": " attached to a battery pack"}, "idx": 3, "task_name": "openbookqa", "doc_id": 326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "continuation": " in sticks"}, "idx": 0, "task_name": "openbookqa", "doc_id": 327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "continuation": " in pebbles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "continuation": " in sand"}, "idx": 2, "task_name": "openbookqa", "doc_id": 327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "continuation": " in spiked plants"}, "idx": 3, "task_name": "openbookqa", "doc_id": 327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why might a polar bear grow white hair?\nAnswer:", "continuation": " look fancy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why might a polar bear grow white hair?\nAnswer:", "continuation": " random"}, "idx": 1, "task_name": "openbookqa", "doc_id": 328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why might a polar bear grow white hair?\nAnswer:", "continuation": " blend in"}, "idx": 2, "task_name": "openbookqa", "doc_id": 328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Why might a polar bear grow white hair?\nAnswer:", "continuation": " stand out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Xylem\nAnswer:", "continuation": " discourages pests from landing on leaves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Xylem\nAnswer:", "continuation": " allows plants to move carbon dioxide from root to stems"}, "idx": 1, "task_name": "openbookqa", "doc_id": 329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Xylem\nAnswer:", "continuation": " carries seedlings from roots to leaves"}, "idx": 2, "task_name": "openbookqa", "doc_id": 329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Xylem\nAnswer:", "continuation": " allows plants to move rain thru their systems"}, "idx": 3, "task_name": "openbookqa", "doc_id": 329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A food that is a source of heat is\nAnswer:", "continuation": " ramen"}, "idx": 0, "task_name": "openbookqa", "doc_id": 330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A food that is a source of heat is\nAnswer:", "continuation": " salad"}, "idx": 1, "task_name": "openbookqa", "doc_id": 330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A food that is a source of heat is\nAnswer:", "continuation": " ice cream"}, "idx": 2, "task_name": "openbookqa", "doc_id": 330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A food that is a source of heat is\nAnswer:", "continuation": " sushi"}, "idx": 3, "task_name": "openbookqa", "doc_id": 330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When heat is added to something\nAnswer:", "continuation": " contaminates may be destroyed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When heat is added to something\nAnswer:", "continuation": " bacterial can grow more rapidly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When heat is added to something\nAnswer:", "continuation": " viruses may be picked up"}, "idx": 2, "task_name": "openbookqa", "doc_id": 331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When heat is added to something\nAnswer:", "continuation": " the thing loses energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The heart is an example of\nAnswer:", "continuation": " a part of the nervous system"}, "idx": 0, "task_name": "openbookqa", "doc_id": 332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The heart is an example of\nAnswer:", "continuation": " an organ that filters toxins"}, "idx": 1, "task_name": "openbookqa", "doc_id": 332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The heart is an example of\nAnswer:", "continuation": " a self-healing protector from germs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The heart is an example of\nAnswer:", "continuation": " something protected by the skeletal system"}, "idx": 3, "task_name": "openbookqa", "doc_id": 332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Prey are eaten by\nAnswer:", "continuation": " an animal herded by sheep dogs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Prey are eaten by\nAnswer:", "continuation": " the animal with a starring role in Bambi"}, "idx": 1, "task_name": "openbookqa", "doc_id": 333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Prey are eaten by\nAnswer:", "continuation": " animals known for their memory"}, "idx": 2, "task_name": "openbookqa", "doc_id": 333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Prey are eaten by\nAnswer:", "continuation": " the fastest mammal with four legs"}, "idx": 3, "task_name": "openbookqa", "doc_id": 333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is warm-blooded just like\nAnswer:", "continuation": " a snake"}, "idx": 0, "task_name": "openbookqa", "doc_id": 334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is warm-blooded just like\nAnswer:", "continuation": " a cardinal"}, "idx": 1, "task_name": "openbookqa", "doc_id": 334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is warm-blooded just like\nAnswer:", "continuation": " a spider"}, "idx": 2, "task_name": "openbookqa", "doc_id": 334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is warm-blooded just like\nAnswer:", "continuation": " a scorpion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A flashlight will need this in order to radiate photons:\nAnswer:", "continuation": " radiation"}, "idx": 0, "task_name": "openbookqa", "doc_id": 335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A flashlight will need this in order to radiate photons:\nAnswer:", "continuation": " acoustic energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A flashlight will need this in order to radiate photons:\nAnswer:", "continuation": " vibrations"}, "idx": 2, "task_name": "openbookqa", "doc_id": 335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A flashlight will need this in order to radiate photons:\nAnswer:", "continuation": " electron flow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: to find out how fast you are going you first need to know\nAnswer:", "continuation": " where you're going"}, "idx": 0, "task_name": "openbookqa", "doc_id": 336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: to find out how fast you are going you first need to know\nAnswer:", "continuation": " distance traveled"}, "idx": 1, "task_name": "openbookqa", "doc_id": 336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: to find out how fast you are going you first need to know\nAnswer:", "continuation": " distance to travel"}, "idx": 2, "task_name": "openbookqa", "doc_id": 336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: to find out how fast you are going you first need to know\nAnswer:", "continuation": " home location"}, "idx": 3, "task_name": "openbookqa", "doc_id": 336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "continuation": " weaken"}, "idx": 0, "task_name": "openbookqa", "doc_id": 337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "continuation": " strengthen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "continuation": " coexist"}, "idx": 2, "task_name": "openbookqa", "doc_id": 337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "continuation": " thrive"}, "idx": 3, "task_name": "openbookqa", "doc_id": 337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "continuation": " a dinner jacket"}, "idx": 0, "task_name": "openbookqa", "doc_id": 338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "continuation": " a diving suit"}, "idx": 1, "task_name": "openbookqa", "doc_id": 338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "continuation": " a warm coat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "continuation": " a dress suit"}, "idx": 3, "task_name": "openbookqa", "doc_id": 338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do polar bears survive the cold?\nAnswer:", "continuation": " B and D"}, "idx": 0, "task_name": "openbookqa", "doc_id": 339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do polar bears survive the cold?\nAnswer:", "continuation": " Double Fur Coats"}, "idx": 1, "task_name": "openbookqa", "doc_id": 339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do polar bears survive the cold?\nAnswer:", "continuation": " Cold blooded"}, "idx": 2, "task_name": "openbookqa", "doc_id": 339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do polar bears survive the cold?\nAnswer:", "continuation": " Compact ears"}, "idx": 3, "task_name": "openbookqa", "doc_id": 339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A solid is likely to form in extreme\nAnswer:", "continuation": " floods"}, "idx": 0, "task_name": "openbookqa", "doc_id": 340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A solid is likely to form in extreme\nAnswer:", "continuation": " wind"}, "idx": 1, "task_name": "openbookqa", "doc_id": 340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A solid is likely to form in extreme\nAnswer:", "continuation": " chill"}, "idx": 2, "task_name": "openbookqa", "doc_id": 340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A solid is likely to form in extreme\nAnswer:", "continuation": " rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "continuation": " catching prey"}, "idx": 0, "task_name": "openbookqa", "doc_id": 341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "continuation": " aquatic speed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "continuation": " flying"}, "idx": 2, "task_name": "openbookqa", "doc_id": 341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "continuation": " walking"}, "idx": 3, "task_name": "openbookqa", "doc_id": 341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of fire giving off light?\nAnswer:", "continuation": " an oven is preheated and the pilot light is lit"}, "idx": 0, "task_name": "openbookqa", "doc_id": 342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of fire giving off light?\nAnswer:", "continuation": " a match is lit to light a cigarette"}, "idx": 1, "task_name": "openbookqa", "doc_id": 342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of fire giving off light?\nAnswer:", "continuation": " a lit candle in a window signalling to someone"}, "idx": 2, "task_name": "openbookqa", "doc_id": 342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of fire giving off light?\nAnswer:", "continuation": " a fire that was put out to send smoke signals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The respiratory system works by\nAnswer:", "continuation": " directing oxygen from lungs to other organs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The respiratory system works by\nAnswer:", "continuation": " pushing air through lungs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The respiratory system works by\nAnswer:", "continuation": " moving air in a room"}, "idx": 2, "task_name": "openbookqa", "doc_id": 343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The respiratory system works by\nAnswer:", "continuation": " making air quality better"}, "idx": 3, "task_name": "openbookqa", "doc_id": 343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals have more fat\nAnswer:", "continuation": " in the ocean"}, "idx": 0, "task_name": "openbookqa", "doc_id": 344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals have more fat\nAnswer:", "continuation": " in human homes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals have more fat\nAnswer:", "continuation": " in landfills"}, "idx": 2, "task_name": "openbookqa", "doc_id": 344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals have more fat\nAnswer:", "continuation": " in polar areas"}, "idx": 3, "task_name": "openbookqa", "doc_id": 344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A stick bug uses what to protect itself from predators?\nAnswer:", "continuation": " poison"}, "idx": 0, "task_name": "openbookqa", "doc_id": 345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A stick bug uses what to protect itself from predators?\nAnswer:", "continuation": " its appearance"}, "idx": 1, "task_name": "openbookqa", "doc_id": 345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A stick bug uses what to protect itself from predators?\nAnswer:", "continuation": " speed"}, "idx": 2, "task_name": "openbookqa", "doc_id": 345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A stick bug uses what to protect itself from predators?\nAnswer:", "continuation": " hearing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Corn is sometimes used to make\nAnswer:", "continuation": " a simple alcohol"}, "idx": 0, "task_name": "openbookqa", "doc_id": 346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Corn is sometimes used to make\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Corn is sometimes used to make\nAnswer:", "continuation": " glass"}, "idx": 2, "task_name": "openbookqa", "doc_id": 346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Corn is sometimes used to make\nAnswer:", "continuation": " milk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "continuation": " heat energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "continuation": " light energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "continuation": " color energy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "continuation": " color transfusion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Little puppies are a result of:\nAnswer:", "continuation": " reproduction ?"}, "idx": 0, "task_name": "openbookqa", "doc_id": 348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Little puppies are a result of:\nAnswer:", "continuation": " pet store sale"}, "idx": 1, "task_name": "openbookqa", "doc_id": 348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Little puppies are a result of:\nAnswer:", "continuation": " a begging child"}, "idx": 2, "task_name": "openbookqa", "doc_id": 348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Little puppies are a result of:\nAnswer:", "continuation": " evolution"}, "idx": 3, "task_name": "openbookqa", "doc_id": 348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would a Jersey most likely be fed?\nAnswer:", "continuation": " hamburger"}, "idx": 0, "task_name": "openbookqa", "doc_id": 349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would a Jersey most likely be fed?\nAnswer:", "continuation": " moles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would a Jersey most likely be fed?\nAnswer:", "continuation": " alfalfa"}, "idx": 2, "task_name": "openbookqa", "doc_id": 349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would a Jersey most likely be fed?\nAnswer:", "continuation": " cow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these energy sources generates the least amount of pollution?\nAnswer:", "continuation": " coal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these energy sources generates the least amount of pollution?\nAnswer:", "continuation": " windmill"}, "idx": 1, "task_name": "openbookqa", "doc_id": 350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these energy sources generates the least amount of pollution?\nAnswer:", "continuation": " lithium batteries"}, "idx": 2, "task_name": "openbookqa", "doc_id": 350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these energy sources generates the least amount of pollution?\nAnswer:", "continuation": " gasoline"}, "idx": 3, "task_name": "openbookqa", "doc_id": 350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grass snakes live in what?\nAnswer:", "continuation": " trees"}, "idx": 0, "task_name": "openbookqa", "doc_id": 351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grass snakes live in what?\nAnswer:", "continuation": " mountains"}, "idx": 1, "task_name": "openbookqa", "doc_id": 351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grass snakes live in what?\nAnswer:", "continuation": " lakes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Grass snakes live in what?\nAnswer:", "continuation": " turf"}, "idx": 3, "task_name": "openbookqa", "doc_id": 351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cephalopod ink is by octopuses to\nAnswer:", "continuation": " mate"}, "idx": 0, "task_name": "openbookqa", "doc_id": 352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cephalopod ink is by octopuses to\nAnswer:", "continuation": " feed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cephalopod ink is by octopuses to\nAnswer:", "continuation": " hide"}, "idx": 2, "task_name": "openbookqa", "doc_id": 352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cephalopod ink is by octopuses to\nAnswer:", "continuation": " play"}, "idx": 3, "task_name": "openbookqa", "doc_id": 352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tapping a drumstick to a drum will\nAnswer:", "continuation": " reverberate when touched together"}, "idx": 0, "task_name": "openbookqa", "doc_id": 353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tapping a drumstick to a drum will\nAnswer:", "continuation": " vibrate when next to each other"}, "idx": 1, "task_name": "openbookqa", "doc_id": 353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tapping a drumstick to a drum will\nAnswer:", "continuation": " shake around when near"}, "idx": 2, "task_name": "openbookqa", "doc_id": 353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Tapping a drumstick to a drum will\nAnswer:", "continuation": " put each other down"}, "idx": 3, "task_name": "openbookqa", "doc_id": 353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Snow is more likely to fall two months before\nAnswer:", "continuation": " June"}, "idx": 0, "task_name": "openbookqa", "doc_id": 354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Snow is more likely to fall two months before\nAnswer:", "continuation": " March"}, "idx": 1, "task_name": "openbookqa", "doc_id": 354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Snow is more likely to fall two months before\nAnswer:", "continuation": " September"}, "idx": 2, "task_name": "openbookqa", "doc_id": 354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Snow is more likely to fall two months before\nAnswer:", "continuation": " December"}, "idx": 3, "task_name": "openbookqa", "doc_id": 354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "continuation": " Scorpion"}, "idx": 0, "task_name": "openbookqa", "doc_id": 355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "continuation": " House Fly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "continuation": " Cricket"}, "idx": 2, "task_name": "openbookqa", "doc_id": 355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "continuation": " Moth"}, "idx": 3, "task_name": "openbookqa", "doc_id": 355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dead plants are easier to find in\nAnswer:", "continuation": " January"}, "idx": 0, "task_name": "openbookqa", "doc_id": 356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dead plants are easier to find in\nAnswer:", "continuation": " July"}, "idx": 1, "task_name": "openbookqa", "doc_id": 356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dead plants are easier to find in\nAnswer:", "continuation": " May"}, "idx": 2, "task_name": "openbookqa", "doc_id": 356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Dead plants are easier to find in\nAnswer:", "continuation": " September"}, "idx": 3, "task_name": "openbookqa", "doc_id": 356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The harder a child pushes a toy car\nAnswer:", "continuation": " decreases the distance it will travel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The harder a child pushes a toy car\nAnswer:", "continuation": " the further it will roll across the floor"}, "idx": 1, "task_name": "openbookqa", "doc_id": 357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The harder a child pushes a toy car\nAnswer:", "continuation": " the quicker the child will want to play with another toy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The harder a child pushes a toy car\nAnswer:", "continuation": " determines how long the child with play with it"}, "idx": 3, "task_name": "openbookqa", "doc_id": 357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fossil fuels\nAnswer:", "continuation": " come from old age"}, "idx": 0, "task_name": "openbookqa", "doc_id": 358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fossil fuels\nAnswer:", "continuation": " come from expired life"}, "idx": 1, "task_name": "openbookqa", "doc_id": 358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fossil fuels\nAnswer:", "continuation": " take two years to create"}, "idx": 2, "task_name": "openbookqa", "doc_id": 358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Fossil fuels\nAnswer:", "continuation": " are created in a year"}, "idx": 3, "task_name": "openbookqa", "doc_id": 358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "continuation": " a room to have overhead lights"}, "idx": 0, "task_name": "openbookqa", "doc_id": 359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "continuation": " night on Earth to be dimly lit"}, "idx": 1, "task_name": "openbookqa", "doc_id": 359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "continuation": " plastic stars to decorate a ceiling"}, "idx": 2, "task_name": "openbookqa", "doc_id": 359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "continuation": " a person to be the star of a show"}, "idx": 3, "task_name": "openbookqa", "doc_id": 359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion is caused by different kinds of\nAnswer:", "continuation": " soil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion is caused by different kinds of\nAnswer:", "continuation": " fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion is caused by different kinds of\nAnswer:", "continuation": " rocks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Erosion is caused by different kinds of\nAnswer:", "continuation": " weather"}, "idx": 3, "task_name": "openbookqa", "doc_id": 360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Respiration is a\nAnswer:", "continuation": " happens for some species"}, "idx": 0, "task_name": "openbookqa", "doc_id": 361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Respiration is a\nAnswer:", "continuation": " happens for only land dwelling mammals"}, "idx": 1, "task_name": "openbookqa", "doc_id": 361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Respiration is a\nAnswer:", "continuation": " occurs for only sea creatures"}, "idx": 2, "task_name": "openbookqa", "doc_id": 361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Respiration is a\nAnswer:", "continuation": " commonality among all animals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "continuation": " magical"}, "idx": 0, "task_name": "openbookqa", "doc_id": 362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "continuation": " closed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "continuation": " broken"}, "idx": 2, "task_name": "openbookqa", "doc_id": 362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "continuation": " open"}, "idx": 3, "task_name": "openbookqa", "doc_id": 362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears live in\nAnswer:", "continuation": " frosty environments"}, "idx": 0, "task_name": "openbookqa", "doc_id": 363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears live in\nAnswer:", "continuation": " tepid environments"}, "idx": 1, "task_name": "openbookqa", "doc_id": 363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears live in\nAnswer:", "continuation": " warm environments"}, "idx": 2, "task_name": "openbookqa", "doc_id": 363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears live in\nAnswer:", "continuation": " tropical environments"}, "idx": 3, "task_name": "openbookqa", "doc_id": 363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be more likely to attract a magnet?\nAnswer:", "continuation": " a plastic zipper"}, "idx": 0, "task_name": "openbookqa", "doc_id": 364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be more likely to attract a magnet?\nAnswer:", "continuation": " flowing water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be more likely to attract a magnet?\nAnswer:", "continuation": " a car engine"}, "idx": 2, "task_name": "openbookqa", "doc_id": 364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would be more likely to attract a magnet?\nAnswer:", "continuation": " A wooden desk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What constitutes a frog's diet?\nAnswer:", "continuation": " it eats all plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What constitutes a frog's diet?\nAnswer:", "continuation": " it will eat dogs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What constitutes a frog's diet?\nAnswer:", "continuation": " it only eats burgers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What constitutes a frog's diet?\nAnswer:", "continuation": " it chomps on insects"}, "idx": 3, "task_name": "openbookqa", "doc_id": 365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of the digestive system digesting food for the body?\nAnswer:", "continuation": " a man eating nachos then getting food poisoning"}, "idx": 0, "task_name": "openbookqa", "doc_id": 366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of the digestive system digesting food for the body?\nAnswer:", "continuation": " a baby drinking formula then needing a diaper change"}, "idx": 1, "task_name": "openbookqa", "doc_id": 366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of the digestive system digesting food for the body?\nAnswer:", "continuation": " a cat eating food then throwing it up"}, "idx": 2, "task_name": "openbookqa", "doc_id": 366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of the digestive system digesting food for the body?\nAnswer:", "continuation": " a horse licking a salt lick"}, "idx": 3, "task_name": "openbookqa", "doc_id": 366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The body is negatively impacted by\nAnswer:", "continuation": " white blood cells"}, "idx": 0, "task_name": "openbookqa", "doc_id": 367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The body is negatively impacted by\nAnswer:", "continuation": " vitamins"}, "idx": 1, "task_name": "openbookqa", "doc_id": 367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The body is negatively impacted by\nAnswer:", "continuation": " rotavirus"}, "idx": 2, "task_name": "openbookqa", "doc_id": 367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The body is negatively impacted by\nAnswer:", "continuation": " nasal decongestants"}, "idx": 3, "task_name": "openbookqa", "doc_id": 367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "continuation": " run wire through currants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "continuation": " run a continuous current"}, "idx": 1, "task_name": "openbookqa", "doc_id": 368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "continuation": " run around the wire"}, "idx": 2, "task_name": "openbookqa", "doc_id": 368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "continuation": " currently run wire through"}, "idx": 3, "task_name": "openbookqa", "doc_id": 368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a place has experienced flooding, what could be responsible?\nAnswer:", "continuation": " all of these"}, "idx": 0, "task_name": "openbookqa", "doc_id": 369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a place has experienced flooding, what could be responsible?\nAnswer:", "continuation": " there has been excess condensed water vapor"}, "idx": 1, "task_name": "openbookqa", "doc_id": 369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a place has experienced flooding, what could be responsible?\nAnswer:", "continuation": " the water lacks oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a place has experienced flooding, what could be responsible?\nAnswer:", "continuation": " the local deities are angry"}, "idx": 3, "task_name": "openbookqa", "doc_id": 369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of reproduction?\nAnswer:", "continuation": " farming"}, "idx": 0, "task_name": "openbookqa", "doc_id": 370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of reproduction?\nAnswer:", "continuation": " egg depositing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of reproduction?\nAnswer:", "continuation": " flying"}, "idx": 2, "task_name": "openbookqa", "doc_id": 370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What is an example of reproduction?\nAnswer:", "continuation": " walking"}, "idx": 3, "task_name": "openbookqa", "doc_id": 370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A place that is snowy has a large amount of\nAnswer:", "continuation": " wind"}, "idx": 0, "task_name": "openbookqa", "doc_id": 371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A place that is snowy has a large amount of\nAnswer:", "continuation": " storms"}, "idx": 1, "task_name": "openbookqa", "doc_id": 371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A place that is snowy has a large amount of\nAnswer:", "continuation": " frozen water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A place that is snowy has a large amount of\nAnswer:", "continuation": " rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ideal abode for crickets is\nAnswer:", "continuation": " a small potted plant in a house"}, "idx": 0, "task_name": "openbookqa", "doc_id": 372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ideal abode for crickets is\nAnswer:", "continuation": " a green and lush tree and plant packed area"}, "idx": 1, "task_name": "openbookqa", "doc_id": 372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ideal abode for crickets is\nAnswer:", "continuation": " a briny and warm body of water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ideal abode for crickets is\nAnswer:", "continuation": " a area surrounded by spider webs"}, "idx": 3, "task_name": "openbookqa", "doc_id": 372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbon steel is always what?\nAnswer:", "continuation": " attractive to various objects that contain iron"}, "idx": 0, "task_name": "openbookqa", "doc_id": 373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbon steel is always what?\nAnswer:", "continuation": " pleasant with a magnetic personality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbon steel is always what?\nAnswer:", "continuation": " made up of iron and pieces of magnets"}, "idx": 2, "task_name": "openbookqa", "doc_id": 373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbon steel is always what?\nAnswer:", "continuation": " hard as a magnetizing rod"}, "idx": 3, "task_name": "openbookqa", "doc_id": 373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "continuation": " it is a mammal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "continuation": " calling it a bird is wrong"}, "idx": 1, "task_name": "openbookqa", "doc_id": 374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "continuation": " it is capable of reproducing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "continuation": " minerals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "continuation": " mist"}, "idx": 1, "task_name": "openbookqa", "doc_id": 375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "continuation": " clouds"}, "idx": 2, "task_name": "openbookqa", "doc_id": 375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "continuation": " water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: as you get closer to something it begins to\nAnswer:", "continuation": " shrinks down to nothing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: as you get closer to something it begins to\nAnswer:", "continuation": " grow in size visually"}, "idx": 1, "task_name": "openbookqa", "doc_id": 376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: as you get closer to something it begins to\nAnswer:", "continuation": " show a large shadow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: as you get closer to something it begins to\nAnswer:", "continuation": " rotate in a clockwise direction"}, "idx": 3, "task_name": "openbookqa", "doc_id": 376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a storm\nAnswer:", "continuation": " ponds may dry out"}, "idx": 0, "task_name": "openbookqa", "doc_id": 377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a storm\nAnswer:", "continuation": " flowers will wilt and wither"}, "idx": 1, "task_name": "openbookqa", "doc_id": 377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a storm\nAnswer:", "continuation": " creek beds may be spilling over"}, "idx": 2, "task_name": "openbookqa", "doc_id": 377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: After a storm\nAnswer:", "continuation": " drinking water will be in short supply"}, "idx": 3, "task_name": "openbookqa", "doc_id": 377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could I use as biofuel\nAnswer:", "continuation": " Gold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could I use as biofuel\nAnswer:", "continuation": " Car"}, "idx": 1, "task_name": "openbookqa", "doc_id": 378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could I use as biofuel\nAnswer:", "continuation": " Diamonds"}, "idx": 2, "task_name": "openbookqa", "doc_id": 378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could I use as biofuel\nAnswer:", "continuation": " Pine Needles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is hiding from a predator?\nAnswer:", "continuation": " a tadpole losing its tail as it grows"}, "idx": 0, "task_name": "openbookqa", "doc_id": 379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is hiding from a predator?\nAnswer:", "continuation": " an angler fish using its Esca to lure another fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is hiding from a predator?\nAnswer:", "continuation": " an octopus mimicking the color and texture of a rocky outcrop"}, "idx": 2, "task_name": "openbookqa", "doc_id": 379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal is hiding from a predator?\nAnswer:", "continuation": " a great white shark breaching the water's surface"}, "idx": 3, "task_name": "openbookqa", "doc_id": 379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "continuation": " skating on a rough surface"}, "idx": 0, "task_name": "openbookqa", "doc_id": 380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "continuation": " a full bag swung in circles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "continuation": " a computer powering on"}, "idx": 2, "task_name": "openbookqa", "doc_id": 380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "continuation": " a baker stirring batter"}, "idx": 3, "task_name": "openbookqa", "doc_id": 380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the night sky shows very far away what\nAnswer:", "continuation": " clumps of flaming gas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the night sky shows very far away what\nAnswer:", "continuation": " tidal waves washing over beaches"}, "idx": 1, "task_name": "openbookqa", "doc_id": 381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the night sky shows very far away what\nAnswer:", "continuation": " aircraft falling towards collision"}, "idx": 2, "task_name": "openbookqa", "doc_id": 381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the night sky shows very far away what\nAnswer:", "continuation": " party balloons tied to houses"}, "idx": 3, "task_name": "openbookqa", "doc_id": 381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which would you likely find inside a beach ball?\nAnswer:", "continuation": " cheese"}, "idx": 0, "task_name": "openbookqa", "doc_id": 382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which would you likely find inside a beach ball?\nAnswer:", "continuation": " steam"}, "idx": 1, "task_name": "openbookqa", "doc_id": 382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which would you likely find inside a beach ball?\nAnswer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which would you likely find inside a beach ball?\nAnswer:", "continuation": " air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Two fridge decorations when touched back to back\nAnswer:", "continuation": " shove each other away"}, "idx": 0, "task_name": "openbookqa", "doc_id": 383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Two fridge decorations when touched back to back\nAnswer:", "continuation": " are attracted to each other"}, "idx": 1, "task_name": "openbookqa", "doc_id": 383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Two fridge decorations when touched back to back\nAnswer:", "continuation": " have very little reaction"}, "idx": 2, "task_name": "openbookqa", "doc_id": 383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Two fridge decorations when touched back to back\nAnswer:", "continuation": " are reflective when together"}, "idx": 3, "task_name": "openbookqa", "doc_id": 383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Runoff happens because of\nAnswer:", "continuation": " birds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Runoff happens because of\nAnswer:", "continuation": " cattails"}, "idx": 1, "task_name": "openbookqa", "doc_id": 384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Runoff happens because of\nAnswer:", "continuation": " people"}, "idx": 2, "task_name": "openbookqa", "doc_id": 384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Runoff happens because of\nAnswer:", "continuation": " fish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments are generally\nAnswer:", "continuation": " sweltering"}, "idx": 0, "task_name": "openbookqa", "doc_id": 385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments are generally\nAnswer:", "continuation": " arctic like"}, "idx": 1, "task_name": "openbookqa", "doc_id": 385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments are generally\nAnswer:", "continuation": " lush"}, "idx": 2, "task_name": "openbookqa", "doc_id": 385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Desert environments are generally\nAnswer:", "continuation": " frigid"}, "idx": 3, "task_name": "openbookqa", "doc_id": 385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Are deserts characterized by high sunshine?\nAnswer:", "continuation": " they get low sunlight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Are deserts characterized by high sunshine?\nAnswer:", "continuation": " deserts get surplus sun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Are deserts characterized by high sunshine?\nAnswer:", "continuation": " deserts get little sun"}, "idx": 2, "task_name": "openbookqa", "doc_id": 386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Are deserts characterized by high sunshine?\nAnswer:", "continuation": " deserts are always cloudy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water conservation could be a survival tactic in\nAnswer:", "continuation": " The Appalachian Mountains"}, "idx": 0, "task_name": "openbookqa", "doc_id": 387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water conservation could be a survival tactic in\nAnswer:", "continuation": " New York City"}, "idx": 1, "task_name": "openbookqa", "doc_id": 387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water conservation could be a survival tactic in\nAnswer:", "continuation": " The Amazon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water conservation could be a survival tactic in\nAnswer:", "continuation": " The Gobi Desert"}, "idx": 3, "task_name": "openbookqa", "doc_id": 387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Objects used to hold sheets of paper together are often\nAnswer:", "continuation": " large"}, "idx": 0, "task_name": "openbookqa", "doc_id": 388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Objects used to hold sheets of paper together are often\nAnswer:", "continuation": " wooden"}, "idx": 1, "task_name": "openbookqa", "doc_id": 388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Objects used to hold sheets of paper together are often\nAnswer:", "continuation": " ferromagnetic"}, "idx": 2, "task_name": "openbookqa", "doc_id": 388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Objects used to hold sheets of paper together are often\nAnswer:", "continuation": " electronic"}, "idx": 3, "task_name": "openbookqa", "doc_id": 388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "continuation": " energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "continuation": " fun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "continuation": " taste"}, "idx": 2, "task_name": "openbookqa", "doc_id": 389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "continuation": " soil"}, "idx": 3, "task_name": "openbookqa", "doc_id": 389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bear cub learns to stay away from unknown bears because\nAnswer:", "continuation": " they are much bigger than the cub"}, "idx": 0, "task_name": "openbookqa", "doc_id": 390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bear cub learns to stay away from unknown bears because\nAnswer:", "continuation": " the other bears look like its mother"}, "idx": 1, "task_name": "openbookqa", "doc_id": 390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bear cub learns to stay away from unknown bears because\nAnswer:", "continuation": " their mother teaches them to keep their distance"}, "idx": 2, "task_name": "openbookqa", "doc_id": 390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A bear cub learns to stay away from unknown bears because\nAnswer:", "continuation": " the unknown bears look harmless"}, "idx": 3, "task_name": "openbookqa", "doc_id": 390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a person driving to work in which of these is most likely to lose control?\nAnswer:", "continuation": " a dry cobblestone road"}, "idx": 0, "task_name": "openbookqa", "doc_id": 391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a person driving to work in which of these is most likely to lose control?\nAnswer:", "continuation": " a sleet covered highway"}, "idx": 1, "task_name": "openbookqa", "doc_id": 391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a person driving to work in which of these is most likely to lose control?\nAnswer:", "continuation": " a dry paved road"}, "idx": 2, "task_name": "openbookqa", "doc_id": 391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: a person driving to work in which of these is most likely to lose control?\nAnswer:", "continuation": " a dry gravel road"}, "idx": 3, "task_name": "openbookqa", "doc_id": 391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The only creature with offspring that is hatched, of these, is the\nAnswer:", "continuation": " squirrel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The only creature with offspring that is hatched, of these, is the\nAnswer:", "continuation": " swallow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The only creature with offspring that is hatched, of these, is the\nAnswer:", "continuation": " mink"}, "idx": 2, "task_name": "openbookqa", "doc_id": 392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The only creature with offspring that is hatched, of these, is the\nAnswer:", "continuation": " bat"}, "idx": 3, "task_name": "openbookqa", "doc_id": 392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What element is prevalent in a plateau?\nAnswer:", "continuation": " helium"}, "idx": 0, "task_name": "openbookqa", "doc_id": 393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What element is prevalent in a plateau?\nAnswer:", "continuation": " krypton"}, "idx": 1, "task_name": "openbookqa", "doc_id": 393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What element is prevalent in a plateau?\nAnswer:", "continuation": " silicon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What element is prevalent in a plateau?\nAnswer:", "continuation": " neon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a city tears down a park in a city, the park\nAnswer:", "continuation": " is removed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a city tears down a park in a city, the park\nAnswer:", "continuation": " is renewed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a city tears down a park in a city, the park\nAnswer:", "continuation": " is retrieved"}, "idx": 2, "task_name": "openbookqa", "doc_id": 394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When a city tears down a park in a city, the park\nAnswer:", "continuation": " is restored"}, "idx": 3, "task_name": "openbookqa", "doc_id": 394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds\nAnswer:", "continuation": " are useless shells that need to be discarded"}, "idx": 0, "task_name": "openbookqa", "doc_id": 395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds\nAnswer:", "continuation": " store extra bits of chlorophyll"}, "idx": 1, "task_name": "openbookqa", "doc_id": 395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds\nAnswer:", "continuation": " need to be mashed to grow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Seeds\nAnswer:", "continuation": " aid in feeding what grows from them"}, "idx": 3, "task_name": "openbookqa", "doc_id": 395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "continuation": " consumption"}, "idx": 0, "task_name": "openbookqa", "doc_id": 396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "continuation": " cinematography"}, "idx": 1, "task_name": "openbookqa", "doc_id": 396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "continuation": " direction"}, "idx": 2, "task_name": "openbookqa", "doc_id": 396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "continuation": " production"}, "idx": 3, "task_name": "openbookqa", "doc_id": 396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The spring season brings\nAnswer:", "continuation": " Bees"}, "idx": 0, "task_name": "openbookqa", "doc_id": 397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The spring season brings\nAnswer:", "continuation": " Snow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The spring season brings\nAnswer:", "continuation": " More Oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The spring season brings\nAnswer:", "continuation": " Dust"}, "idx": 3, "task_name": "openbookqa", "doc_id": 397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Kinetic energy can be found in objects that move, such as\nAnswer:", "continuation": " flower pots on a wagon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Kinetic energy can be found in objects that move, such as\nAnswer:", "continuation": " cars that are in a lot"}, "idx": 1, "task_name": "openbookqa", "doc_id": 398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Kinetic energy can be found in objects that move, such as\nAnswer:", "continuation": " kids that are sleeping soundly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Kinetic energy can be found in objects that move, such as\nAnswer:", "continuation": " skateboards that are ridden all day"}, "idx": 3, "task_name": "openbookqa", "doc_id": 398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these is the quickest to go visiting from our world?\nAnswer:", "continuation": " none of these"}, "idx": 0, "task_name": "openbookqa", "doc_id": 399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these is the quickest to go visiting from our world?\nAnswer:", "continuation": " a trip to mars"}, "idx": 1, "task_name": "openbookqa", "doc_id": 399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these is the quickest to go visiting from our world?\nAnswer:", "continuation": " a trip to the moon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these is the quickest to go visiting from our world?\nAnswer:", "continuation": " a trip to the northern star"}, "idx": 3, "task_name": "openbookqa", "doc_id": 399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals died after the removal of a\nAnswer:", "continuation": " bush"}, "idx": 0, "task_name": "openbookqa", "doc_id": 400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals died after the removal of a\nAnswer:", "continuation": " street"}, "idx": 1, "task_name": "openbookqa", "doc_id": 400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals died after the removal of a\nAnswer:", "continuation": " house"}, "idx": 2, "task_name": "openbookqa", "doc_id": 400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Animals died after the removal of a\nAnswer:", "continuation": " city"}, "idx": 3, "task_name": "openbookqa", "doc_id": 400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to go running at night, what can I use as a reflector?\nAnswer:", "continuation": " A black shirt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to go running at night, what can I use as a reflector?\nAnswer:", "continuation": " Kitchen foil"}, "idx": 1, "task_name": "openbookqa", "doc_id": 401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to go running at night, what can I use as a reflector?\nAnswer:", "continuation": " Sunglasses"}, "idx": 2, "task_name": "openbookqa", "doc_id": 401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If I want to go running at night, what can I use as a reflector?\nAnswer:", "continuation": " A megaphone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the closest star to our planet delivers solar energy to the planet\nAnswer:", "continuation": " maybe"}, "idx": 0, "task_name": "openbookqa", "doc_id": 402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the closest star to our planet delivers solar energy to the planet\nAnswer:", "continuation": " all of these"}, "idx": 1, "task_name": "openbookqa", "doc_id": 402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the closest star to our planet delivers solar energy to the planet\nAnswer:", "continuation": " this is sure"}, "idx": 2, "task_name": "openbookqa", "doc_id": 402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: the closest star to our planet delivers solar energy to the planet\nAnswer:", "continuation": " this is uncertain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "continuation": " produce energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "continuation": " use heat energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "continuation": " burn energy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "continuation": " fuel the world"}, "idx": 3, "task_name": "openbookqa", "doc_id": 403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "continuation": " wasp"}, "idx": 0, "task_name": "openbookqa", "doc_id": 404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "continuation": " bee"}, "idx": 1, "task_name": "openbookqa", "doc_id": 404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "continuation": " scorpion"}, "idx": 2, "task_name": "openbookqa", "doc_id": 404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "continuation": " butterfly"}, "idx": 3, "task_name": "openbookqa", "doc_id": 404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A renewable resource is\nAnswer:", "continuation": " fossil fuel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A renewable resource is\nAnswer:", "continuation": " turbine produced electricity"}, "idx": 1, "task_name": "openbookqa", "doc_id": 405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A renewable resource is\nAnswer:", "continuation": " copper"}, "idx": 2, "task_name": "openbookqa", "doc_id": 405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A renewable resource is\nAnswer:", "continuation": " coal lumps"}, "idx": 3, "task_name": "openbookqa", "doc_id": 405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "continuation": " would cease existing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "continuation": " would be troubled"}, "idx": 1, "task_name": "openbookqa", "doc_id": 406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "continuation": " would be thriving"}, "idx": 2, "task_name": "openbookqa", "doc_id": 406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "continuation": " would be endangered"}, "idx": 3, "task_name": "openbookqa", "doc_id": 406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Acid can be used to make a new\nAnswer:", "continuation": " light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Acid can be used to make a new\nAnswer:", "continuation": " substance"}, "idx": 1, "task_name": "openbookqa", "doc_id": 407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Acid can be used to make a new\nAnswer:", "continuation": " electricity"}, "idx": 2, "task_name": "openbookqa", "doc_id": 407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Acid can be used to make a new\nAnswer:", "continuation": " sound"}, "idx": 3, "task_name": "openbookqa", "doc_id": 407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Global warming is lowering the world's amount of\nAnswer:", "continuation": " hurricanes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Global warming is lowering the world's amount of\nAnswer:", "continuation": " ocean levels"}, "idx": 1, "task_name": "openbookqa", "doc_id": 408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Global warming is lowering the world's amount of\nAnswer:", "continuation": " carbon dioxide"}, "idx": 2, "task_name": "openbookqa", "doc_id": 408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Global warming is lowering the world's amount of\nAnswer:", "continuation": " ice"}, "idx": 3, "task_name": "openbookqa", "doc_id": 408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Echolocation can't detect an object's\nAnswer:", "continuation": " distance"}, "idx": 0, "task_name": "openbookqa", "doc_id": 409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Echolocation can't detect an object's\nAnswer:", "continuation": " shape"}, "idx": 1, "task_name": "openbookqa", "doc_id": 409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Echolocation can't detect an object's\nAnswer:", "continuation": " size"}, "idx": 2, "task_name": "openbookqa", "doc_id": 409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Echolocation can't detect an object's\nAnswer:", "continuation": " temperature"}, "idx": 3, "task_name": "openbookqa", "doc_id": 409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What material has already broken down?\nAnswer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What material has already broken down?\nAnswer:", "continuation": " glass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What material has already broken down?\nAnswer:", "continuation": " boulders"}, "idx": 2, "task_name": "openbookqa", "doc_id": 410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What material has already broken down?\nAnswer:", "continuation": " sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which beverage would dissolve solids the best?\nAnswer:", "continuation": " A glass of ice-cold water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which beverage would dissolve solids the best?\nAnswer:", "continuation": " A boiling hot mug of tea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which beverage would dissolve solids the best?\nAnswer:", "continuation": " A cup of warm milk"}, "idx": 2, "task_name": "openbookqa", "doc_id": 411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which beverage would dissolve solids the best?\nAnswer:", "continuation": " A room temperature glass of water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal has live births?\nAnswer:", "continuation": " poodle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal has live births?\nAnswer:", "continuation": " hummingbird"}, "idx": 1, "task_name": "openbookqa", "doc_id": 412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal has live births?\nAnswer:", "continuation": " crocodile"}, "idx": 2, "task_name": "openbookqa", "doc_id": 412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which animal has live births?\nAnswer:", "continuation": " trout"}, "idx": 3, "task_name": "openbookqa", "doc_id": 412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Quartz crystals are made up of\nAnswer:", "continuation": " majic"}, "idx": 0, "task_name": "openbookqa", "doc_id": 413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Quartz crystals are made up of\nAnswer:", "continuation": " hexagons"}, "idx": 1, "task_name": "openbookqa", "doc_id": 413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Quartz crystals are made up of\nAnswer:", "continuation": " octogons"}, "idx": 2, "task_name": "openbookqa", "doc_id": 413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Quartz crystals are made up of\nAnswer:", "continuation": " water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "continuation": " water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "continuation": " nutrients"}, "idx": 1, "task_name": "openbookqa", "doc_id": 414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "continuation": " mitochondria"}, "idx": 2, "task_name": "openbookqa", "doc_id": 414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "continuation": " gas"}, "idx": 3, "task_name": "openbookqa", "doc_id": 414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Did pasteurization get invented by Thomas Edison?\nAnswer:", "continuation": " negative"}, "idx": 0, "task_name": "openbookqa", "doc_id": 415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Did pasteurization get invented by Thomas Edison?\nAnswer:", "continuation": " positive"}, "idx": 1, "task_name": "openbookqa", "doc_id": 415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Did pasteurization get invented by Thomas Edison?\nAnswer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Did pasteurization get invented by Thomas Edison?\nAnswer:", "continuation": " maybe it was"}, "idx": 3, "task_name": "openbookqa", "doc_id": 415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A glass of water can undergo a chemical change by adding\nAnswer:", "continuation": " a cup of salt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A glass of water can undergo a chemical change by adding\nAnswer:", "continuation": " a cup of dirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A glass of water can undergo a chemical change by adding\nAnswer:", "continuation": " a cup of water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A glass of water can undergo a chemical change by adding\nAnswer:", "continuation": " a cup of ice"}, "idx": 3, "task_name": "openbookqa", "doc_id": 416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water levels may decrease on cloudless days because\nAnswer:", "continuation": " water is warmer than the air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water levels may decrease on cloudless days because\nAnswer:", "continuation": " air is warmer than water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water levels may decrease on cloudless days because\nAnswer:", "continuation": " moisture is pulled upwards"}, "idx": 2, "task_name": "openbookqa", "doc_id": 417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water levels may decrease on cloudless days because\nAnswer:", "continuation": " moisture always tries to rise"}, "idx": 3, "task_name": "openbookqa", "doc_id": 417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To change an object's shape\nAnswer:", "continuation": " rip off a corner portion"}, "idx": 0, "task_name": "openbookqa", "doc_id": 418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To change an object's shape\nAnswer:", "continuation": " lay it flat on a table"}, "idx": 1, "task_name": "openbookqa", "doc_id": 418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To change an object's shape\nAnswer:", "continuation": " color the edges of it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: To change an object's shape\nAnswer:", "continuation": " add a piece of tape to it"}, "idx": 3, "task_name": "openbookqa", "doc_id": 418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "continuation": " reports of tornadoes in the area"}, "idx": 0, "task_name": "openbookqa", "doc_id": 419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "continuation": " a dog running across the highway behind Steve's car"}, "idx": 1, "task_name": "openbookqa", "doc_id": 419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "continuation": " a sudden fog moving into the area"}, "idx": 2, "task_name": "openbookqa", "doc_id": 419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "continuation": " ice forming on the road"}, "idx": 3, "task_name": "openbookqa", "doc_id": 419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: DNA is a vehicle for passing\nAnswer:", "continuation": " clothes types"}, "idx": 0, "task_name": "openbookqa", "doc_id": 420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: DNA is a vehicle for passing\nAnswer:", "continuation": " school grades"}, "idx": 1, "task_name": "openbookqa", "doc_id": 420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: DNA is a vehicle for passing\nAnswer:", "continuation": " elbow size"}, "idx": 2, "task_name": "openbookqa", "doc_id": 420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: DNA is a vehicle for passing\nAnswer:", "continuation": " language and dialect"}, "idx": 3, "task_name": "openbookqa", "doc_id": 420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "continuation": " food"}, "idx": 0, "task_name": "openbookqa", "doc_id": 421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "continuation": " sunlight"}, "idx": 1, "task_name": "openbookqa", "doc_id": 421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "continuation": " gas"}, "idx": 2, "task_name": "openbookqa", "doc_id": 421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "continuation": " salt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In general, how many times per month is there a full moon?\nAnswer:", "continuation": " twice"}, "idx": 0, "task_name": "openbookqa", "doc_id": 422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In general, how many times per month is there a full moon?\nAnswer:", "continuation": " three times"}, "idx": 1, "task_name": "openbookqa", "doc_id": 422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In general, how many times per month is there a full moon?\nAnswer:", "continuation": " once"}, "idx": 2, "task_name": "openbookqa", "doc_id": 422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In general, how many times per month is there a full moon?\nAnswer:", "continuation": " four times"}, "idx": 3, "task_name": "openbookqa", "doc_id": 422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of object does light bounce off of?\nAnswer:", "continuation": " tadpole"}, "idx": 0, "task_name": "openbookqa", "doc_id": 423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of object does light bounce off of?\nAnswer:", "continuation": " any object"}, "idx": 1, "task_name": "openbookqa", "doc_id": 423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of object does light bounce off of?\nAnswer:", "continuation": " item that reflects"}, "idx": 2, "task_name": "openbookqa", "doc_id": 423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What kind of object does light bounce off of?\nAnswer:", "continuation": " black hole"}, "idx": 3, "task_name": "openbookqa", "doc_id": 423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "continuation": " the season has been quite dry"}, "idx": 0, "task_name": "openbookqa", "doc_id": 424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "continuation": " There has been a lot of rain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "continuation": " snakes shelter under the brush"}, "idx": 2, "task_name": "openbookqa", "doc_id": 424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "continuation": " People have been walking by the brush on the trails"}, "idx": 3, "task_name": "openbookqa", "doc_id": 424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "continuation": " the body of the pot is of high temperature"}, "idx": 0, "task_name": "openbookqa", "doc_id": 425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "continuation": " the body of the pot is cold"}, "idx": 1, "task_name": "openbookqa", "doc_id": 425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "continuation": " the body of the pot is wet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal is more difficult for predators to see in water?\nAnswer:", "continuation": " a fish"}, "idx": 0, "task_name": "openbookqa", "doc_id": 426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal is more difficult for predators to see in water?\nAnswer:", "continuation": " a duck"}, "idx": 1, "task_name": "openbookqa", "doc_id": 426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal is more difficult for predators to see in water?\nAnswer:", "continuation": " an octopus"}, "idx": 2, "task_name": "openbookqa", "doc_id": 426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What animal is more difficult for predators to see in water?\nAnswer:", "continuation": " a crab"}, "idx": 3, "task_name": "openbookqa", "doc_id": 426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A wedge requires\nAnswer:", "continuation": " electrical energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A wedge requires\nAnswer:", "continuation": " chemical energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A wedge requires\nAnswer:", "continuation": " mechanical energy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A wedge requires\nAnswer:", "continuation": " heat energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In solid phase matter has a/an\nAnswer:", "continuation": " concrete configuration"}, "idx": 0, "task_name": "openbookqa", "doc_id": 428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In solid phase matter has a/an\nAnswer:", "continuation": " ambiguous form"}, "idx": 1, "task_name": "openbookqa", "doc_id": 428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In solid phase matter has a/an\nAnswer:", "continuation": " shapeless form"}, "idx": 2, "task_name": "openbookqa", "doc_id": 428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: In solid phase matter has a/an\nAnswer:", "continuation": " radioactive glow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "continuation": " takes a trip to the desert"}, "idx": 0, "task_name": "openbookqa", "doc_id": 429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "continuation": " climbs a tall mountain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "continuation": " travels to the seaside"}, "idx": 2, "task_name": "openbookqa", "doc_id": 429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "continuation": " visits a remote jungle"}, "idx": 3, "task_name": "openbookqa", "doc_id": 429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "continuation": " adding salt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "continuation": " adding litter"}, "idx": 1, "task_name": "openbookqa", "doc_id": 430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "continuation": " adding sand"}, "idx": 2, "task_name": "openbookqa", "doc_id": 430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "continuation": " adding water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The appropriate place to put this item is the recycling bin\nAnswer:", "continuation": " used motor oil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The appropriate place to put this item is the recycling bin\nAnswer:", "continuation": " used soda can"}, "idx": 1, "task_name": "openbookqa", "doc_id": 431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The appropriate place to put this item is the recycling bin\nAnswer:", "continuation": " used Styrofoam plates"}, "idx": 2, "task_name": "openbookqa", "doc_id": 431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The appropriate place to put this item is the recycling bin\nAnswer:", "continuation": " left over medicine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How many times would someone change the page of a calendar in a year?\nAnswer:", "continuation": " 13"}, "idx": 0, "task_name": "openbookqa", "doc_id": 432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How many times would someone change the page of a calendar in a year?\nAnswer:", "continuation": " 12"}, "idx": 1, "task_name": "openbookqa", "doc_id": 432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How many times would someone change the page of a calendar in a year?\nAnswer:", "continuation": " 15"}, "idx": 2, "task_name": "openbookqa", "doc_id": 432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How many times would someone change the page of a calendar in a year?\nAnswer:", "continuation": " 14"}, "idx": 3, "task_name": "openbookqa", "doc_id": 432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A car has the least speed if it\nAnswer:", "continuation": " is heavy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A car has the least speed if it\nAnswer:", "continuation": " is large"}, "idx": 1, "task_name": "openbookqa", "doc_id": 433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A car has the least speed if it\nAnswer:", "continuation": " is turned off"}, "idx": 2, "task_name": "openbookqa", "doc_id": 433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A car has the least speed if it\nAnswer:", "continuation": " is small"}, "idx": 3, "task_name": "openbookqa", "doc_id": 433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which musical instrument is the same type as a guitar?\nAnswer:", "continuation": " flute"}, "idx": 0, "task_name": "openbookqa", "doc_id": 434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which musical instrument is the same type as a guitar?\nAnswer:", "continuation": " cello"}, "idx": 1, "task_name": "openbookqa", "doc_id": 434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which musical instrument is the same type as a guitar?\nAnswer:", "continuation": " drum"}, "idx": 2, "task_name": "openbookqa", "doc_id": 434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which musical instrument is the same type as a guitar?\nAnswer:", "continuation": " trumpet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The balance result will be number of\nAnswer:", "continuation": " kilowatts"}, "idx": 0, "task_name": "openbookqa", "doc_id": 435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The balance result will be number of\nAnswer:", "continuation": " kilobytes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The balance result will be number of\nAnswer:", "continuation": " kilograms"}, "idx": 2, "task_name": "openbookqa", "doc_id": 435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The balance result will be number of\nAnswer:", "continuation": " kilometers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a place where a human might live?\nAnswer:", "continuation": " igloo"}, "idx": 0, "task_name": "openbookqa", "doc_id": 436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a place where a human might live?\nAnswer:", "continuation": " cloud"}, "idx": 1, "task_name": "openbookqa", "doc_id": 436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a place where a human might live?\nAnswer:", "continuation": " Mars"}, "idx": 2, "task_name": "openbookqa", "doc_id": 436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is a place where a human might live?\nAnswer:", "continuation": " the Moon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage is when an organism does what?\nAnswer:", "continuation": " reconfigure appearance to blend in"}, "idx": 0, "task_name": "openbookqa", "doc_id": 437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage is when an organism does what?\nAnswer:", "continuation": " hides its young to avoid prey"}, "idx": 1, "task_name": "openbookqa", "doc_id": 437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage is when an organism does what?\nAnswer:", "continuation": " changes its shape to appear larger"}, "idx": 2, "task_name": "openbookqa", "doc_id": 437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage is when an organism does what?\nAnswer:", "continuation": " buries itself to disappear momentarily"}, "idx": 3, "task_name": "openbookqa", "doc_id": 437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "continuation": " xylem"}, "idx": 0, "task_name": "openbookqa", "doc_id": 438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "continuation": " luck"}, "idx": 1, "task_name": "openbookqa", "doc_id": 438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "continuation": " magic"}, "idx": 2, "task_name": "openbookqa", "doc_id": 438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "continuation": " dirt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How does a microscope make small things appear?\nAnswer:", "continuation": " humongous"}, "idx": 0, "task_name": "openbookqa", "doc_id": 439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How does a microscope make small things appear?\nAnswer:", "continuation": " transparent"}, "idx": 1, "task_name": "openbookqa", "doc_id": 439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How does a microscope make small things appear?\nAnswer:", "continuation": " discolored"}, "idx": 2, "task_name": "openbookqa", "doc_id": 439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How does a microscope make small things appear?\nAnswer:", "continuation": " distorted"}, "idx": 3, "task_name": "openbookqa", "doc_id": 439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water can turn to vapor\nAnswer:", "continuation": " when a pot of water is placed on an off stove burner"}, "idx": 0, "task_name": "openbookqa", "doc_id": 440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water can turn to vapor\nAnswer:", "continuation": " when placing water in a freezer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water can turn to vapor\nAnswer:", "continuation": " when boiling eggs on a stove top"}, "idx": 2, "task_name": "openbookqa", "doc_id": 440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Water can turn to vapor\nAnswer:", "continuation": " when placed in a room temperature setting"}, "idx": 3, "task_name": "openbookqa", "doc_id": 440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "continuation": " white light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "continuation": " conversion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "continuation": " heat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "continuation": " sound"}, "idx": 3, "task_name": "openbookqa", "doc_id": 441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "continuation": " seconds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "continuation": " days"}, "idx": 1, "task_name": "openbookqa", "doc_id": 442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "continuation": " weeks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "continuation": " years"}, "idx": 3, "task_name": "openbookqa", "doc_id": 442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage can be used by animals for hunting\nAnswer:", "continuation": " water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage can be used by animals for hunting\nAnswer:", "continuation": " trees"}, "idx": 1, "task_name": "openbookqa", "doc_id": 443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage can be used by animals for hunting\nAnswer:", "continuation": " air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Camouflage can be used by animals for hunting\nAnswer:", "continuation": " meals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "continuation": " broccoli"}, "idx": 0, "task_name": "openbookqa", "doc_id": 444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "continuation": " meat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "continuation": " celery"}, "idx": 2, "task_name": "openbookqa", "doc_id": 444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "continuation": " toast"}, "idx": 3, "task_name": "openbookqa", "doc_id": 444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Hand dryers can also be used to\nAnswer:", "continuation": " keep cold drinks cool"}, "idx": 0, "task_name": "openbookqa", "doc_id": 445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Hand dryers can also be used to\nAnswer:", "continuation": " dry out clothes after coming in from the rain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Hand dryers can also be used to\nAnswer:", "continuation": " hydrate your face and hands"}, "idx": 2, "task_name": "openbookqa", "doc_id": 445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Hand dryers can also be used to\nAnswer:", "continuation": " make a damp rag damper"}, "idx": 3, "task_name": "openbookqa", "doc_id": 445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears require\nAnswer:", "continuation": " a tropical environment"}, "idx": 0, "task_name": "openbookqa", "doc_id": 446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears require\nAnswer:", "continuation": " a frigid environment"}, "idx": 1, "task_name": "openbookqa", "doc_id": 446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears require\nAnswer:", "continuation": " a tepid environment"}, "idx": 2, "task_name": "openbookqa", "doc_id": 446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Polar bears require\nAnswer:", "continuation": " a warm environment"}, "idx": 3, "task_name": "openbookqa", "doc_id": 446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A measurement of time that is less than a minute is a\nAnswer:", "continuation": " day"}, "idx": 0, "task_name": "openbookqa", "doc_id": 447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A measurement of time that is less than a minute is a\nAnswer:", "continuation": " minute"}, "idx": 1, "task_name": "openbookqa", "doc_id": 447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A measurement of time that is less than a minute is a\nAnswer:", "continuation": " hour"}, "idx": 2, "task_name": "openbookqa", "doc_id": 447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A measurement of time that is less than a minute is a\nAnswer:", "continuation": " second"}, "idx": 3, "task_name": "openbookqa", "doc_id": 447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will be more available in an area when rainfall increases?\nAnswer:", "continuation": " fire"}, "idx": 0, "task_name": "openbookqa", "doc_id": 448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will be more available in an area when rainfall increases?\nAnswer:", "continuation": " air"}, "idx": 1, "task_name": "openbookqa", "doc_id": 448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will be more available in an area when rainfall increases?\nAnswer:", "continuation": " dirt"}, "idx": 2, "task_name": "openbookqa", "doc_id": 448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What will be more available in an area when rainfall increases?\nAnswer:", "continuation": " H2O"}, "idx": 3, "task_name": "openbookqa", "doc_id": 448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When does the first quarter phase of the moon occur?\nAnswer:", "continuation": " when you cannot see the moon in the sky at night"}, "idx": 0, "task_name": "openbookqa", "doc_id": 449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When does the first quarter phase of the moon occur?\nAnswer:", "continuation": " after the first phase of the lunar month"}, "idx": 1, "task_name": "openbookqa", "doc_id": 449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When does the first quarter phase of the moon occur?\nAnswer:", "continuation": " after a blue moon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When does the first quarter phase of the moon occur?\nAnswer:", "continuation": " during the full moon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ice cube placed in sunlight will\nAnswer:", "continuation": " shrink"}, "idx": 0, "task_name": "openbookqa", "doc_id": 450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ice cube placed in sunlight will\nAnswer:", "continuation": " change color"}, "idx": 1, "task_name": "openbookqa", "doc_id": 450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ice cube placed in sunlight will\nAnswer:", "continuation": " grow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An ice cube placed in sunlight will\nAnswer:", "continuation": " freeze"}, "idx": 3, "task_name": "openbookqa", "doc_id": 450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "continuation": " destroying"}, "idx": 0, "task_name": "openbookqa", "doc_id": 451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "continuation": " conserving"}, "idx": 1, "task_name": "openbookqa", "doc_id": 451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "continuation": " losing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "continuation": " squandering"}, "idx": 3, "task_name": "openbookqa", "doc_id": 451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The skeletal system protects which of these?\nAnswer:", "continuation": " liver"}, "idx": 0, "task_name": "openbookqa", "doc_id": 452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The skeletal system protects which of these?\nAnswer:", "continuation": " eyelashes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The skeletal system protects which of these?\nAnswer:", "continuation": " finger nails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The skeletal system protects which of these?\nAnswer:", "continuation": " blood vessels"}, "idx": 3, "task_name": "openbookqa", "doc_id": 452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What has more gravity force than Earth but less than the sun?\nAnswer:", "continuation": " Jupiter"}, "idx": 0, "task_name": "openbookqa", "doc_id": 453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What has more gravity force than Earth but less than the sun?\nAnswer:", "continuation": " the moon"}, "idx": 1, "task_name": "openbookqa", "doc_id": 453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What has more gravity force than Earth but less than the sun?\nAnswer:", "continuation": " a space station"}, "idx": 2, "task_name": "openbookqa", "doc_id": 453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What has more gravity force than Earth but less than the sun?\nAnswer:", "continuation": " a comet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The dam was put under much more stress after the\nAnswer:", "continuation": " party"}, "idx": 0, "task_name": "openbookqa", "doc_id": 454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The dam was put under much more stress after the\nAnswer:", "continuation": " huge rain storm"}, "idx": 1, "task_name": "openbookqa", "doc_id": 454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The dam was put under much more stress after the\nAnswer:", "continuation": " drought"}, "idx": 2, "task_name": "openbookqa", "doc_id": 454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The dam was put under much more stress after the\nAnswer:", "continuation": " breakup."}, "idx": 3, "task_name": "openbookqa", "doc_id": 454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "continuation": " CO2, water, and argon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "continuation": " sunlight, oxygen, and fertilizer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "continuation": " CO2, H20, and cloudy skies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "continuation": " CO2, H20, and sun rays"}, "idx": 3, "task_name": "openbookqa", "doc_id": 455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a nail is Fe, that nail is\nAnswer:", "continuation": " foreign"}, "idx": 0, "task_name": "openbookqa", "doc_id": 456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a nail is Fe, that nail is\nAnswer:", "continuation": " atomic 26"}, "idx": 1, "task_name": "openbookqa", "doc_id": 456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a nail is Fe, that nail is\nAnswer:", "continuation": " nickel"}, "idx": 2, "task_name": "openbookqa", "doc_id": 456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: If a nail is Fe, that nail is\nAnswer:", "continuation": " atomic 12"}, "idx": 3, "task_name": "openbookqa", "doc_id": 456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when a circle is torn it is\nAnswer:", "continuation": " doubled"}, "idx": 0, "task_name": "openbookqa", "doc_id": 457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when a circle is torn it is\nAnswer:", "continuation": " changed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when a circle is torn it is\nAnswer:", "continuation": " a smaller circle"}, "idx": 2, "task_name": "openbookqa", "doc_id": 457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: when a circle is torn it is\nAnswer:", "continuation": " a square"}, "idx": 3, "task_name": "openbookqa", "doc_id": 457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " leaves to remain on branches"}, "idx": 0, "task_name": "openbookqa", "doc_id": 458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " trees to stand perfectly still"}, "idx": 1, "task_name": "openbookqa", "doc_id": 458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " dunes at the beach to be depleted"}, "idx": 2, "task_name": "openbookqa", "doc_id": 458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Wind can cause\nAnswer:", "continuation": " still waters on the ocean"}, "idx": 3, "task_name": "openbookqa", "doc_id": 458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What happens as water levels rise?\nAnswer:", "continuation": " fish swim more"}, "idx": 0, "task_name": "openbookqa", "doc_id": 459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What happens as water levels rise?\nAnswer:", "continuation": " homes are built"}, "idx": 1, "task_name": "openbookqa", "doc_id": 459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What happens as water levels rise?\nAnswer:", "continuation": " land is taller"}, "idx": 2, "task_name": "openbookqa", "doc_id": 459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What happens as water levels rise?\nAnswer:", "continuation": " beaches shrink"}, "idx": 3, "task_name": "openbookqa", "doc_id": 459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An increase in an object's temperature occurs when\nAnswer:", "continuation": " an orange is placed in a refrigerator"}, "idx": 0, "task_name": "openbookqa", "doc_id": 460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An increase in an object's temperature occurs when\nAnswer:", "continuation": " a steak is removed from the freezer to defrost"}, "idx": 1, "task_name": "openbookqa", "doc_id": 460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An increase in an object's temperature occurs when\nAnswer:", "continuation": " a glass of water is moved from counter top to dinner table"}, "idx": 2, "task_name": "openbookqa", "doc_id": 460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An increase in an object's temperature occurs when\nAnswer:", "continuation": " an ice tray is placed in a freezer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sousaphone\nAnswer:", "continuation": " is ancient"}, "idx": 0, "task_name": "openbookqa", "doc_id": 461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sousaphone\nAnswer:", "continuation": " is a frog"}, "idx": 1, "task_name": "openbookqa", "doc_id": 461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sousaphone\nAnswer:", "continuation": " makes deep noises"}, "idx": 2, "task_name": "openbookqa", "doc_id": 461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A sousaphone\nAnswer:", "continuation": " is a smartphone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cooked lobster is\nAnswer:", "continuation": " inedible"}, "idx": 0, "task_name": "openbookqa", "doc_id": 462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cooked lobster is\nAnswer:", "continuation": " cold"}, "idx": 1, "task_name": "openbookqa", "doc_id": 462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cooked lobster is\nAnswer:", "continuation": " dead"}, "idx": 2, "task_name": "openbookqa", "doc_id": 462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A cooked lobster is\nAnswer:", "continuation": " green"}, "idx": 3, "task_name": "openbookqa", "doc_id": 462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal might pant\nAnswer:", "continuation": " on a sunny day"}, "idx": 0, "task_name": "openbookqa", "doc_id": 463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal might pant\nAnswer:", "continuation": " during a rain storm"}, "idx": 1, "task_name": "openbookqa", "doc_id": 463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal might pant\nAnswer:", "continuation": " when it is snowing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An animal might pant\nAnswer:", "continuation": " during the night time"}, "idx": 3, "task_name": "openbookqa", "doc_id": 463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carnivores\nAnswer:", "continuation": " eat foliage and vegetables exclusively"}, "idx": 0, "task_name": "openbookqa", "doc_id": 464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carnivores\nAnswer:", "continuation": " are the bottom of the food chain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carnivores\nAnswer:", "continuation": " require prey to survive"}, "idx": 2, "task_name": "openbookqa", "doc_id": 464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Carnivores\nAnswer:", "continuation": " require carbon dioxide to survive"}, "idx": 3, "task_name": "openbookqa", "doc_id": 464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The light that appears dimmest is\nAnswer:", "continuation": " the light in the hall"}, "idx": 0, "task_name": "openbookqa", "doc_id": 465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The light that appears dimmest is\nAnswer:", "continuation": " a light in the room"}, "idx": 1, "task_name": "openbookqa", "doc_id": 465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The light that appears dimmest is\nAnswer:", "continuation": " a star outside the window"}, "idx": 2, "task_name": "openbookqa", "doc_id": 465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The light that appears dimmest is\nAnswer:", "continuation": " a streetlight outside the window"}, "idx": 3, "task_name": "openbookqa", "doc_id": 465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant will grow strong if it has\nAnswer:", "continuation": " love"}, "idx": 0, "task_name": "openbookqa", "doc_id": 466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant will grow strong if it has\nAnswer:", "continuation": " heat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant will grow strong if it has\nAnswer:", "continuation": " earth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant will grow strong if it has\nAnswer:", "continuation": " sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: loose soil can be caused by one of these\nAnswer:", "continuation": " a koala sitting on a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: loose soil can be caused by one of these\nAnswer:", "continuation": " none of these"}, "idx": 1, "task_name": "openbookqa", "doc_id": 467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: loose soil can be caused by one of these\nAnswer:", "continuation": " a worm burrowing through the earth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: loose soil can be caused by one of these\nAnswer:", "continuation": " a bird flying through the air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An instinctual behavior is\nAnswer:", "continuation": " dogs rolling over on command"}, "idx": 0, "task_name": "openbookqa", "doc_id": 468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An instinctual behavior is\nAnswer:", "continuation": " frogs returning to the ponds were they hatched to lay eggs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An instinctual behavior is\nAnswer:", "continuation": " birds mimicking human speech"}, "idx": 2, "task_name": "openbookqa", "doc_id": 468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An instinctual behavior is\nAnswer:", "continuation": " seals clapping for treats from trainers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do plants reproduce?\nAnswer:", "continuation": " seeds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do plants reproduce?\nAnswer:", "continuation": " stem"}, "idx": 1, "task_name": "openbookqa", "doc_id": 469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do plants reproduce?\nAnswer:", "continuation": " flowers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: How do plants reproduce?\nAnswer:", "continuation": " leaves"}, "idx": 3, "task_name": "openbookqa", "doc_id": 469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What produce pollen and seeds?\nAnswer:", "continuation": " lakes that are frozen over"}, "idx": 0, "task_name": "openbookqa", "doc_id": 470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What produce pollen and seeds?\nAnswer:", "continuation": " things you give a loved one in a bouquet"}, "idx": 1, "task_name": "openbookqa", "doc_id": 470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What produce pollen and seeds?\nAnswer:", "continuation": " various types of animals"}, "idx": 2, "task_name": "openbookqa", "doc_id": 470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What produce pollen and seeds?\nAnswer:", "continuation": " a person that is healthy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would be most ideal for plant root growth?\nAnswer:", "continuation": " a sticky clay soil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would be most ideal for plant root growth?\nAnswer:", "continuation": " soil with worms burrowing around"}, "idx": 1, "task_name": "openbookqa", "doc_id": 471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would be most ideal for plant root growth?\nAnswer:", "continuation": " an arid soil with little looseness"}, "idx": 2, "task_name": "openbookqa", "doc_id": 471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: which of these would be most ideal for plant root growth?\nAnswer:", "continuation": " all of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Having a sense of touch means\nAnswer:", "continuation": " I am the water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Having a sense of touch means\nAnswer:", "continuation": " I am a tree"}, "idx": 1, "task_name": "openbookqa", "doc_id": 472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Having a sense of touch means\nAnswer:", "continuation": " I am an Ant"}, "idx": 2, "task_name": "openbookqa", "doc_id": 472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Having a sense of touch means\nAnswer:", "continuation": " I am the Air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Live birth is exemplified in\nAnswer:", "continuation": " snakes slithering out of eggs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Live birth is exemplified in\nAnswer:", "continuation": " a calf emerging from a mother giraffe"}, "idx": 1, "task_name": "openbookqa", "doc_id": 473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Live birth is exemplified in\nAnswer:", "continuation": " owlets pecking out of their encasement"}, "idx": 2, "task_name": "openbookqa", "doc_id": 473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Live birth is exemplified in\nAnswer:", "continuation": " sea turtles emerging from their shells"}, "idx": 3, "task_name": "openbookqa", "doc_id": 473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cooking peas requires\nAnswer:", "continuation": " fresh briny sea water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cooking peas requires\nAnswer:", "continuation": " an unheated stove top"}, "idx": 1, "task_name": "openbookqa", "doc_id": 474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cooking peas requires\nAnswer:", "continuation": " salt and cayenne pepper"}, "idx": 2, "task_name": "openbookqa", "doc_id": 474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Cooking peas requires\nAnswer:", "continuation": " turning on a stove top"}, "idx": 3, "task_name": "openbookqa", "doc_id": 474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth revolves around\nAnswer:", "continuation": " the moon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth revolves around\nAnswer:", "continuation": " outer space"}, "idx": 1, "task_name": "openbookqa", "doc_id": 475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth revolves around\nAnswer:", "continuation": " another planet"}, "idx": 2, "task_name": "openbookqa", "doc_id": 475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Earth revolves around\nAnswer:", "continuation": " an energy source"}, "idx": 3, "task_name": "openbookqa", "doc_id": 475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A satellite orbits a\nAnswer:", "continuation": " empty space"}, "idx": 0, "task_name": "openbookqa", "doc_id": 476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A satellite orbits a\nAnswer:", "continuation": " ocean"}, "idx": 1, "task_name": "openbookqa", "doc_id": 476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A satellite orbits a\nAnswer:", "continuation": " terrestrial body"}, "idx": 2, "task_name": "openbookqa", "doc_id": 476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A satellite orbits a\nAnswer:", "continuation": " air pocket"}, "idx": 3, "task_name": "openbookqa", "doc_id": 476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "continuation": " they will increase"}, "idx": 0, "task_name": "openbookqa", "doc_id": 477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "continuation": " nothing will happen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "continuation": " they will shrink"}, "idx": 2, "task_name": "openbookqa", "doc_id": 477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "continuation": " they will double"}, "idx": 3, "task_name": "openbookqa", "doc_id": 477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is a heightened threat of landslide in\nAnswer:", "continuation": " the desert"}, "idx": 0, "task_name": "openbookqa", "doc_id": 478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is a heightened threat of landslide in\nAnswer:", "continuation": " The Andes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is a heightened threat of landslide in\nAnswer:", "continuation": " the ocean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: There is a heightened threat of landslide in\nAnswer:", "continuation": " Indiana"}, "idx": 3, "task_name": "openbookqa", "doc_id": 478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of water being an electrical conductor would be what?\nAnswer:", "continuation": " lightening hitting water and organisms inside dying"}, "idx": 0, "task_name": "openbookqa", "doc_id": 479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of water being an electrical conductor would be what?\nAnswer:", "continuation": " standing in a puddle and avoiding being struck by lightening"}, "idx": 1, "task_name": "openbookqa", "doc_id": 479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of water being an electrical conductor would be what?\nAnswer:", "continuation": " standing in a field and getting struck by lightening"}, "idx": 2, "task_name": "openbookqa", "doc_id": 479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: An example of water being an electrical conductor would be what?\nAnswer:", "continuation": " grabbing a fence and being shocked"}, "idx": 3, "task_name": "openbookqa", "doc_id": 479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would Occur once between January 1st and December 31st\nAnswer:", "continuation": " The moons orbit around the year"}, "idx": 0, "task_name": "openbookqa", "doc_id": 480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would Occur once between January 1st and December 31st\nAnswer:", "continuation": " One rotation on mercury"}, "idx": 1, "task_name": "openbookqa", "doc_id": 480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would Occur once between January 1st and December 31st\nAnswer:", "continuation": " The distance between earth and Jupiter when traveling at light speed"}, "idx": 2, "task_name": "openbookqa", "doc_id": 480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What would Occur once between January 1st and December 31st\nAnswer:", "continuation": " A Solar Year on earth"}, "idx": 3, "task_name": "openbookqa", "doc_id": 480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Burning something that reproduces usually will:\nAnswer:", "continuation": " impair its well being in some way"}, "idx": 0, "task_name": "openbookqa", "doc_id": 481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Burning something that reproduces usually will:\nAnswer:", "continuation": " weed out weaker members of the species"}, "idx": 1, "task_name": "openbookqa", "doc_id": 481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Burning something that reproduces usually will:\nAnswer:", "continuation": " speed up its biological functions"}, "idx": 2, "task_name": "openbookqa", "doc_id": 481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Burning something that reproduces usually will:\nAnswer:", "continuation": " increase its population growth"}, "idx": 3, "task_name": "openbookqa", "doc_id": 481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what does a chipmunk do with acorns\nAnswer:", "continuation": " throw them at other chipmunks"}, "idx": 0, "task_name": "openbookqa", "doc_id": 482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what does a chipmunk do with acorns\nAnswer:", "continuation": " leave them where they're found"}, "idx": 1, "task_name": "openbookqa", "doc_id": 482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what does a chipmunk do with acorns\nAnswer:", "continuation": " use them to build shelter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: what does a chipmunk do with acorns\nAnswer:", "continuation": " transfer them to the stomach"}, "idx": 3, "task_name": "openbookqa", "doc_id": 482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "continuation": " cold air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "continuation": " water vapor"}, "idx": 1, "task_name": "openbookqa", "doc_id": 483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "continuation": " very dry"}, "idx": 2, "task_name": "openbookqa", "doc_id": 483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "continuation": " boiling water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant that gets extra minerals such as zinc are probably\nAnswer:", "continuation": " planted in zinc pills"}, "idx": 0, "task_name": "openbookqa", "doc_id": 484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant that gets extra minerals such as zinc are probably\nAnswer:", "continuation": " plated in the sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant that gets extra minerals such as zinc are probably\nAnswer:", "continuation": " placed in good soil"}, "idx": 2, "task_name": "openbookqa", "doc_id": 484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A plant that gets extra minerals such as zinc are probably\nAnswer:", "continuation": " made out of soil"}, "idx": 3, "task_name": "openbookqa", "doc_id": 484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item has a higher altitude?\nAnswer:", "continuation": " Tile Floor"}, "idx": 0, "task_name": "openbookqa", "doc_id": 485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item has a higher altitude?\nAnswer:", "continuation": " Cars"}, "idx": 1, "task_name": "openbookqa", "doc_id": 485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item has a higher altitude?\nAnswer:", "continuation": " A 6'' Man"}, "idx": 2, "task_name": "openbookqa", "doc_id": 485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Which item has a higher altitude?\nAnswer:", "continuation": " A Picture Book"}, "idx": 3, "task_name": "openbookqa", "doc_id": 485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "continuation": " from a live cow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "continuation": " from a live plant"}, "idx": 1, "task_name": "openbookqa", "doc_id": 486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "continuation": " from a volcano cave"}, "idx": 2, "task_name": "openbookqa", "doc_id": 486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "continuation": " from a wild dog"}, "idx": 3, "task_name": "openbookqa", "doc_id": 486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could have covered an organism in order to create a trilobite?\nAnswer:", "continuation": " Grass"}, "idx": 0, "task_name": "openbookqa", "doc_id": 487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could have covered an organism in order to create a trilobite?\nAnswer:", "continuation": " Water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could have covered an organism in order to create a trilobite?\nAnswer:", "continuation": " Snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: What could have covered an organism in order to create a trilobite?\nAnswer:", "continuation": " Sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is more likely to shiver at\nAnswer:", "continuation": " 1 pm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is more likely to shiver at\nAnswer:", "continuation": " 5 am"}, "idx": 1, "task_name": "openbookqa", "doc_id": 488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is more likely to shiver at\nAnswer:", "continuation": " 9 am"}, "idx": 2, "task_name": "openbookqa", "doc_id": 488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A dog is more likely to shiver at\nAnswer:", "continuation": " 6 pm"}, "idx": 3, "task_name": "openbookqa", "doc_id": 488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "continuation": " nourishes small protein bits that need to eat with tiny shakes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "continuation": " providing nourishment which enables some growth to vegetation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "continuation": " mixes carbs into soluble plant matter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "continuation": " makes good vegetable protein"}, "idx": 3, "task_name": "openbookqa", "doc_id": 489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: where might a bunny live?\nAnswer:", "continuation": " a thicket"}, "idx": 0, "task_name": "openbookqa", "doc_id": 490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: where might a bunny live?\nAnswer:", "continuation": " atop palm trees"}, "idx": 1, "task_name": "openbookqa", "doc_id": 490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: where might a bunny live?\nAnswer:", "continuation": " a sewer system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: where might a bunny live?\nAnswer:", "continuation": " a deserted island"}, "idx": 3, "task_name": "openbookqa", "doc_id": 490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "continuation": " it is a predator"}, "idx": 0, "task_name": "openbookqa", "doc_id": 491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "continuation": " it is a vegetarian"}, "idx": 1, "task_name": "openbookqa", "doc_id": 491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "continuation": " it is a freshwater fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "continuation": " it is a producer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "continuation": " oil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "continuation": " deer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "continuation": " bugs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "continuation": " recycled plastic fruit"}, "idx": 3, "task_name": "openbookqa", "doc_id": 492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon was formed by\nAnswer:", "continuation": " a volcano erupting in 1782"}, "idx": 0, "task_name": "openbookqa", "doc_id": 493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon was formed by\nAnswer:", "continuation": " a river named after the 20th state to join the union flowing over time"}, "idx": 1, "task_name": "openbookqa", "doc_id": 493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon was formed by\nAnswer:", "continuation": " a river named after the 38th state to join the union flowing over time"}, "idx": 2, "task_name": "openbookqa", "doc_id": 493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: The Grand Canyon was formed by\nAnswer:", "continuation": " the Great Lakes drying up"}, "idx": 3, "task_name": "openbookqa", "doc_id": 493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "continuation": " UV rays are harmful"}, "idx": 0, "task_name": "openbookqa", "doc_id": 494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "continuation": " sunlight will be fun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "continuation": " the sun is close"}, "idx": 2, "task_name": "openbookqa", "doc_id": 494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "continuation": " the sun is in space"}, "idx": 3, "task_name": "openbookqa", "doc_id": 494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "continuation": " scalds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "continuation": " cools"}, "idx": 1, "task_name": "openbookqa", "doc_id": 495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "continuation": " toasts"}, "idx": 2, "task_name": "openbookqa", "doc_id": 495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "continuation": " freezes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pasta may be cooked in water when\nAnswer:", "continuation": " the water is warm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pasta may be cooked in water when\nAnswer:", "continuation": " the water is on the stove"}, "idx": 1, "task_name": "openbookqa", "doc_id": 496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pasta may be cooked in water when\nAnswer:", "continuation": " water is bubbling from applied warmth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Pasta may be cooked in water when\nAnswer:", "continuation": " the pasta is very fresh"}, "idx": 3, "task_name": "openbookqa", "doc_id": 496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A decrease in diseases\nAnswer:", "continuation": " has no impact on a population"}, "idx": 0, "task_name": "openbookqa", "doc_id": 497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A decrease in diseases\nAnswer:", "continuation": " leads to more sick people"}, "idx": 1, "task_name": "openbookqa", "doc_id": 497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A decrease in diseases\nAnswer:", "continuation": " leads to less sick people"}, "idx": 2, "task_name": "openbookqa", "doc_id": 497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: A decrease in diseases\nAnswer:", "continuation": " leads to an uptick in emergency room visits"}, "idx": 3, "task_name": "openbookqa", "doc_id": 497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "continuation": " insects like big beetles"}, "idx": 0, "task_name": "openbookqa", "doc_id": 498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "continuation": " tiny lifeforms in dirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "continuation": " small mammals living there"}, "idx": 2, "task_name": "openbookqa", "doc_id": 498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "continuation": " a lot of tiny pebbles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "continuation": " cold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 499, "native_id": "7-7", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 499, "native_id": "7-7", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "continuation": " heat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 499, "native_id": "7-7", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Question: When standing miles away from Mount Rushmore\nAnswer: the mountains seem smaller than in photographs\n\nQuestion: When food is reduced in the stomach\nAnswer: nutrients are being deconstructed\n\nQuestion: You can make a telescope with a\nAnswer: mailing tube\n\nQuestion: Poison causes harm to which of the following?\nAnswer: a Tree\n\nQuestion: What happens when mercury is placed in water?\nAnswer: it sinks\n\nQuestion: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "continuation": " humidity"}, "idx": 3, "task_name": "openbookqa", "doc_id": 499, "native_id": "7-7", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Answer:", "continuation": " make more phone calls"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000000, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Answer:", "continuation": " quit eating lunch out"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000000, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Answer:", "continuation": " buy less with monopoly money"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000000, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-343", "query": "Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to\nAnswer:", "choices": ["make more phone calls", "quit eating lunch out", "buy less with monopoly money", "have lunch with friends"], "gold": 1}, "request": {"context": "Answer:", "continuation": " have lunch with friends"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000000, "native_id": "8-343", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a marsh"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000001, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a tundra"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000001, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the plains"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000001, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1129", "query": "Question: There is most likely going to be fog around:\nAnswer:", "choices": ["a marsh", "a tundra", "the plains", "a desert"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a desert"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000001, "native_id": "1129", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " lions"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000002, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " humans"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000002, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " bunnies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000002, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "880", "query": "Question: Predators eat\nAnswer:", "choices": ["lions", "humans", "bunnies", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " grass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000002, "native_id": "880", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Answer:", "continuation": " roots may be split"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000003, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Answer:", "continuation": " roots may begin to die"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000003, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Answer:", "continuation": " parts may break the concrete"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000003, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-999", "query": "Question: Oak tree seeds are planted and a sidewalk is paved right next to that spot, until eventually, the tree is tall and the roots must extend past the sidewalk, which means\nAnswer:", "choices": ["roots may be split", "roots may begin to die", "parts may break the concrete", "roots may fall apart"], "gold": 2}, "request": {"context": "Answer:", "continuation": " roots may fall apart"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000003, "native_id": "7-999", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Answer:", "continuation": " gasoline"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000004, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a power station"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000004, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Answer:", "continuation": " electrical conductors"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000004, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-464", "query": "Question: An electric car runs on electricity via\nAnswer:", "choices": ["gasoline", "a power station", "electrical conductors", "fuel"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fuel"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000004, "native_id": "8-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " oxygen"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000005, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " nitrogen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000005, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " carbon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000005, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-794", "query": "Question: As the rain forest is deforested the atmosphere will increase with\nAnswer:", "choices": ["oxygen", "nitrogen", "carbon", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000005, "native_id": "9-794", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Answer:", "continuation": " gas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000006, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Answer:", "continuation": " hydrogen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000006, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Answer:", "continuation": " ions"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000006, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1163", "query": "Question: an electric car contains a motor that runs on\nAnswer:", "choices": ["gas", "hydrogen", "ions", "plutonium"], "gold": 2}, "request": {"context": "Answer:", "continuation": " plutonium"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000006, "native_id": "9-1163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " moons gravity"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000007, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " human planet rotation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000007, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " global warming"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000007, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-322", "query": "Question: The middle of the day usually involves the bright star nearest to the earth to be straight overhead why?\nAnswer:", "choices": ["moons gravity", "human planet rotation", "global warming", "moon rotation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " moon rotation"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000007, "native_id": "9-322", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Answer:", "continuation": " May"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000008, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Answer:", "continuation": " July"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000008, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Answer:", "continuation": " April"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000008, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1140", "query": "Question: The summer solstice in the northern hemisphere is four months before\nAnswer:", "choices": ["May", "July", "April", "October"], "gold": 3}, "request": {"context": "Answer:", "continuation": " October"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000008, "native_id": "7-1140", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Answer:", "continuation": " microorganisms"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000009, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Answer:", "continuation": " broken stones"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000009, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Answer:", "continuation": " pollution"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000009, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-903", "query": "Question: The main component in dirt is\nAnswer:", "choices": ["microorganisms", "broken stones", "pollution", "bacteria"], "gold": 1}, "request": {"context": "Answer:", "continuation": " bacteria"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000009, "native_id": "7-903", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a cave"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000010, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the ocean."}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000010, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a town"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000010, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-511", "query": "Question: It's easier for human's to survive in:\nAnswer:", "choices": ["a cave", "the ocean.", "a town", "alone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " alone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000010, "native_id": "7-511", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Answer:", "continuation": " fruit"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000011, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Answer:", "continuation": " liquid"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000011, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Answer:", "continuation": " food"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000011, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-937", "query": "Question: A cactus stem is used to store\nAnswer:", "choices": ["fruit", "liquid", "food", "spines"], "gold": 1}, "request": {"context": "Answer:", "continuation": " spines"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000011, "native_id": "9-937", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an eagle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000012, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a cow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000012, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a gecko"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000012, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-201", "query": "Question: A red-tailed hawk is searching for prey. It is most likely to swoop down on\nAnswer:", "choices": ["an eagle", "a cow", "a gecko", "a deer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a deer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000012, "native_id": "8-201", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " parched foliage"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000013, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " torrential rain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000013, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " lush foliage"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000013, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1618", "query": "Question: The chance of wildfires is increased by\nAnswer:", "choices": ["parched foliage", "torrential rain", "lush foliage", "careful fire maintenance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " careful fire maintenance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000013, "native_id": "1618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Answer:", "continuation": " shortage of crops for the food supply"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000014, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an increase in air pollution"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000014, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Answer:", "continuation": " powering the lights in a home"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000014, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "758", "query": "Question: A positive effect of burning biofuel is\nAnswer:", "choices": ["shortage of crops for the food supply", "an increase in air pollution", "powering the lights in a home", "deforestation in the amazon to make room for crops"], "gold": 2}, "request": {"context": "Answer:", "continuation": " deforestation in the amazon to make room for crops"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000014, "native_id": "758", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Answer:", "continuation": " wind power will be expensive"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000015, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Answer:", "continuation": " gas costs will rise"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000015, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Answer:", "continuation": " oil costs will be maintained"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000015, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-414", "query": "Question: As gasoline costs rise, alternative fuels are being used, which means that\nAnswer:", "choices": ["wind power will be expensive", "gas costs will rise", "oil costs will be maintained", "gasoline will be needed less"], "gold": 3}, "request": {"context": "Answer:", "continuation": " gasoline will be needed less"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000015, "native_id": "7-414", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sun grafts"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000016, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sunlight shields"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000016, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Answer:", "continuation": " panels collecting sunlight"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000016, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-675", "query": "Question: A person wants to be able to have more natural power in their home. They choose to cease using a traditional electric company to source this electricity, and so decide to install\nAnswer:", "choices": ["sun grafts", "sunlight shields", "panels collecting sunlight", "solar bees"], "gold": 2}, "request": {"context": "Answer:", "continuation": " solar bees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000016, "native_id": "9-675", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Lake Michigan"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000017, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The Mississippi River"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000017, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Bay of Bengal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000017, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-163", "query": "Question: A Mola Mola might live where?\nAnswer:", "choices": ["Lake Michigan", "The Mississippi River", "Bay of Bengal", "Lake Eerie"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Lake Eerie"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000017, "native_id": "9-163", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " weasel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000018, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " willow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000018, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " mango"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000018, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1032", "query": "Question: Which requires energy to move?\nAnswer:", "choices": ["weasel", "willow", "mango", "poison ivy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " poison ivy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000018, "native_id": "1032", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Answer:", "continuation": " rat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000019, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Answer:", "continuation": " moth"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000019, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Answer:", "continuation": " chimpanzee"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000019, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "889", "query": "Question: An animal that only eats plants is a\nAnswer:", "choices": ["rat", "moth", "chimpanzee", "pig"], "gold": 1}, "request": {"context": "Answer:", "continuation": " pig"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000019, "native_id": "889", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Hanoi"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000020, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Athens"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000020, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Baghdad"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000020, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1160", "query": "Question: There was a lot more water vapor in the air when we went on a trip to\nAnswer:", "choices": ["Hanoi", "Athens", "Baghdad", "Phoenix"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Phoenix"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000020, "native_id": "1160", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " gasoline"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000021, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000021, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000021, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-298", "query": "Question: An example of conservation is avoiding the use of\nAnswer:", "choices": ["gasoline", "air", "snow", "clothes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " clothes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000021, "native_id": "9-298", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Answer:", "continuation": " keeping warm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000022, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Answer:", "continuation": " flying"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000022, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sleeping"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000022, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1189", "query": "Question: What can feathers on Spheniscidae be used for?\nAnswer:", "choices": ["keeping warm", "flying", "sleeping", "eating"], "gold": 0}, "request": {"context": "Answer:", "continuation": " eating"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000022, "native_id": "1189", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Answer:", "continuation": " More fresh water for people to drink"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000023, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Lower Life Expectancy in Countries"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000023, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Answer:", "continuation": " More food for more people"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000023, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-395", "query": "Question: Overpopulation can cause\nAnswer:", "choices": ["More fresh water for people to drink", "Lower Life Expectancy in Countries", "More food for more people", "More space for places to people to live"], "gold": 1}, "request": {"context": "Answer:", "continuation": " More space for places to people to live"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000023, "native_id": "8-395", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Answer:", "continuation": " make a lot of bright lights shine"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000024, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Answer:", "continuation": " summon a brilliant wave of color"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000024, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Answer:", "continuation": " heat up a room"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000024, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-238", "query": "Question: Shining a light through a diamond can\nAnswer:", "choices": ["make a lot of bright lights shine", "summon a brilliant wave of color", "heat up a room", "make a lot of money"], "gold": 1}, "request": {"context": "Answer:", "continuation": " make a lot of money"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000024, "native_id": "7-238", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Answer:", "continuation": " its snout"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000025, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Answer:", "continuation": " its gills"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000025, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Answer:", "continuation": " its nose"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000025, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-372", "query": "Question: If you were attacked by a shark and had to punch it sharply where it pulls in air from, you'd use your hand to make contact with\nAnswer:", "choices": ["its snout", "its gills", "its nose", "its belly"], "gold": 1}, "request": {"context": "Answer:", "continuation": " its belly"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000025, "native_id": "7-372", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a wheel with wet brake pads"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000026, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a wheel without brake pads"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000026, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a wheel with worn brake pads"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000026, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-35", "query": "Question: which of these would stop a car quicker?\nAnswer:", "choices": ["a wheel with wet brake pads", "a wheel without brake pads", "a wheel with worn brake pads", "a wheel with dry brake pads"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a wheel with dry brake pads"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000026, "native_id": "8-35", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the circulatory system"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000027, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the digestive system"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000027, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the school system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000027, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-271", "query": "Question: what system is needed for a body to get its needed supply of the gas humans breathe in?\nAnswer:", "choices": ["the circulatory system", "the digestive system", "the school system", "central nervous system"], "gold": 0}, "request": {"context": "Answer:", "continuation": " central nervous system"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000027, "native_id": "9-271", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Answer:", "continuation": " gone"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000028, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Answer:", "continuation": " breaking"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000028, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Answer:", "continuation": " falling"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000028, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-409", "query": "Question: Every evening a child can look into the night sky and see that the moon is\nAnswer:", "choices": ["gone", "breaking", "falling", "moving upwards"], "gold": 3}, "request": {"context": "Answer:", "continuation": " moving upwards"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000028, "native_id": "9-409", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " wings"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000029, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " ground"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000029, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000029, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "530", "query": "Question: When it's flying, a plane has no friction with the\nAnswer:", "choices": ["wings", "ground", "air", "clouds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " clouds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000029, "native_id": "530", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Answer:", "continuation": " acid rain"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000030, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Answer:", "continuation": " pesticides"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000030, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Answer:", "continuation": " shafts of sunlight"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000030, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1426", "query": "Question: To grow plants require\nAnswer:", "choices": ["acid rain", "pesticides", "shafts of sunlight", "moonbeam rays"], "gold": 2}, "request": {"context": "Answer:", "continuation": " moonbeam rays"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000030, "native_id": "1426", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Answer:", "continuation": " The surroundings they are born in."}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000031, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Answer:", "continuation": " Their parents usual diet."}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000031, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Answer:", "continuation": " Just take a random guess."}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000031, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-466", "query": "Question: What is the best way to guess a babies eye color?\nAnswer:", "choices": ["The surroundings they are born in.", "Their parents usual diet.", "Just take a random guess.", "The genealogy records of their family."], "gold": 3}, "request": {"context": "Answer:", "continuation": " The genealogy records of their family."}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000031, "native_id": "8-466", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Answer:", "continuation": " eagles"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000032, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Answer:", "continuation": " robins"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000032, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Answer:", "continuation": " owls"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000032, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1577", "query": "Question: What animal eats plants?\nAnswer:", "choices": ["eagles", "robins", "owls", "leopards"], "gold": 1}, "request": {"context": "Answer:", "continuation": " leopards"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000032, "native_id": "1577", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The ice caps will completely melt if global warming continues"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000033, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The earth is round"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000033, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The earth revolves around the sun"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000033, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-257", "query": "Question: Which of these is a hypothesis?\nAnswer:", "choices": ["The ice caps will completely melt if global warming continues", "The earth is round", "The earth revolves around the sun", "Gravity causes objects to fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Gravity causes objects to fall"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000033, "native_id": "8-257", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " remains of ancient ponds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000034, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " many collisions that have occured"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000034, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " volcanic explosions over millions of years"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000034, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "378", "query": "Question: What explains the characteristic lunar formations?\nAnswer:", "choices": ["remains of ancient ponds", "many collisions that have occured", "volcanic explosions over millions of years", "sink holes due to the moons porous nature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sink holes due to the moons porous nature"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000034, "native_id": "378", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Water animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000035, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Frogs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000035, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Ants"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000035, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-41", "query": "Question: Tadpoles start their lives as\nAnswer:", "choices": ["Water animals", "Frogs", "Ants", "College Students"], "gold": 0}, "request": {"context": "Answer:", "continuation": " College Students"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000035, "native_id": "8-41", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the apple sitting on a sunny sidewalk"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000036, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the apple in the freezer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000036, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the apple sitting on the shaded stoop"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000036, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-540", "query": "Question: If a person puts out four apples around their home on the same day, the molecules in which apple would be moving the most rapidly?\nAnswer:", "choices": ["the apple sitting on a sunny sidewalk", "the apple in the freezer", "the apple sitting on the shaded stoop", "the apple in a closet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the apple in a closet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000036, "native_id": "9-540", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Answer:", "continuation": " nerves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000037, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Answer:", "continuation": " tibia"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000037, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Answer:", "continuation": " nostril"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000037, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "266", "query": "Question: What is used for sensing visual things?\nAnswer:", "choices": ["nerves", "tibia", "nostril", "cornea"], "gold": 3}, "request": {"context": "Answer:", "continuation": " cornea"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000037, "native_id": "266", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000038, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a telescope"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000038, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " roots"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000038, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1309", "query": "Question: They studied the soil by using\nAnswer:", "choices": ["plants", "a telescope", "roots", "a microscope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a microscope"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000038, "native_id": "1309", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Answer:", "continuation": " refrigerator"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000039, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Answer:", "continuation": " room"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000039, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Answer:", "continuation": " jacket"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000039, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1197", "query": "Question: Bill's arm got cold when he put it inside the\nAnswer:", "choices": ["refrigerator", "room", "jacket", "oven"], "gold": 0}, "request": {"context": "Answer:", "continuation": " oven"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000039, "native_id": "7-1197", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Answer:", "continuation": " transformed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000040, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Answer:", "continuation": " traded"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000040, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Answer:", "continuation": " thrown away"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000040, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-891", "query": "Question: A recyclable material can be\nAnswer:", "choices": ["transformed", "traded", "thrown away", "used more times"], "gold": 3}, "request": {"context": "Answer:", "continuation": " used more times"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000040, "native_id": "7-891", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Mother"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000041, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Fertilization"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000041, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Father"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000041, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1180", "query": "Question: What is different about birth in humans and chickens?\nAnswer:", "choices": ["Mother", "Fertilization", "Father", "the hard shell"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the hard shell"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000041, "native_id": "1180", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Answer:", "continuation": " plastic bags floating in the ocean"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000042, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Answer:", "continuation": " mallard ducks floating on a lake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000042, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cottonwood seeds floating in the air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000042, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1204", "query": "Question: Which of these situations is an example of pollutants?\nAnswer:", "choices": ["plastic bags floating in the ocean", "mallard ducks floating on a lake", "cottonwood seeds floating in the air", "cirrus clouds floating in the sky"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cirrus clouds floating in the sky"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000042, "native_id": "1204", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Answer:", "continuation": " eggs with shells"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000043, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Answer:", "continuation": " nest incubation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000043, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a nest"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000043, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-52", "query": "Question: Human reproduction requires\nAnswer:", "choices": ["eggs with shells", "nest incubation", "a nest", "a womb"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a womb"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000043, "native_id": "7-52", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Answer:", "continuation": " can help you monitor a fever"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000044, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Answer:", "continuation": " indicate levels of mercury in the blood"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000044, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Answer:", "continuation": " read exactly at 98.6 degrees"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000044, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1759", "query": "Question: Thermometers\nAnswer:", "choices": ["can help you monitor a fever", "indicate levels of mercury in the blood", "read exactly at 98.6 degrees", "are used only for babies"], "gold": 0}, "request": {"context": "Answer:", "continuation": " are used only for babies"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000044, "native_id": "1759", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Answer:", "continuation": " someone would turn up the room heater"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000045, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Answer:", "continuation": " someone would turn up the air conditioner"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000045, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Answer:", "continuation": " someone would turn up the music"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000045, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-655", "query": "Question: if the earth was a living room, what can be done to melt the glaciers?\nAnswer:", "choices": ["someone would turn up the room heater", "someone would turn up the air conditioner", "someone would turn up the music", "someone would turn on the light"], "gold": 0}, "request": {"context": "Answer:", "continuation": " someone would turn on the light"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000045, "native_id": "9-655", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " they get bigger"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000046, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " they get smaller"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000046, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " nothing happens"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000046, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "132", "query": "Question: What would happen when balloons heat up?\nAnswer:", "choices": ["they get bigger", "they get smaller", "nothing happens", "they fall down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " they fall down"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000046, "native_id": "132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " expand"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000047, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " melt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000047, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " shrink"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000047, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-79", "query": "Question: A balloon is filled with helium for a party. After the party, the balloons are left in the living room, where a fireplace is heating the room. The balloons\nAnswer:", "choices": ["expand", "melt", "shrink", "fall"], "gold": 0}, "request": {"context": "Answer:", "continuation": " fall"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000047, "native_id": "8-79", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Answer:", "continuation": " desert"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000048, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Answer:", "continuation": " arctic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000048, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Mediterranean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000048, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1835", "query": "Question: Seals are most likely to be found in what type of environment?\nAnswer:", "choices": ["desert", "arctic", "Mediterranean", "tropical"], "gold": 1}, "request": {"context": "Answer:", "continuation": " tropical"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000048, "native_id": "1835", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Answer:", "continuation": " killed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000049, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Answer:", "continuation": " hurt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000049, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Answer:", "continuation": " born"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000049, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-149", "query": "Question: When the eggs hatch, the offspring are\nAnswer:", "choices": ["killed", "hurt", "born", "cold"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cold"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000049, "native_id": "9-149", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a bear or person"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000050, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a bear or shark"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000050, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a bear or lion"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000050, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "695", "query": "Question: Some berries may be eaten by\nAnswer:", "choices": ["a bear or person", "a bear or shark", "a bear or lion", "a bear or wolf"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a bear or wolf"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000050, "native_id": "695", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Answer:", "continuation": " weekly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000051, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Answer:", "continuation": " monthly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000051, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Answer:", "continuation": " annually"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000051, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-179", "query": "Question: A person has a chance to experience an equinox\nAnswer:", "choices": ["weekly", "monthly", "annually", "biannually"], "gold": 3}, "request": {"context": "Answer:", "continuation": " biannually"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000051, "native_id": "8-179", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " strain the resources of an ecosystem"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000052, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cause boundless growth of resources"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000052, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " lead to extinction of the organism"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000052, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-50", "query": "Question: Overpopulation of an organism can\nAnswer:", "choices": ["strain the resources of an ecosystem", "cause boundless growth of resources", "lead to extinction of the organism", "cause the ecosystem to flourish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cause the ecosystem to flourish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000052, "native_id": "7-50", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Answer:", "continuation": " high risk lifestyle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000053, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Answer:", "continuation": " restaurant food"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000053, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Answer:", "continuation": " business trip"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000053, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "508", "query": "Question: To improve health, what is a good strategy?\nAnswer:", "choices": ["high risk lifestyle", "restaurant food", "business trip", "a spa trip"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a spa trip"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000053, "native_id": "508", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Answer:", "continuation": " date of birth"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000054, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Answer:", "continuation": " shirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000054, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Answer:", "continuation": " number of toenails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000054, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1674", "query": "Question: A girl and her mom have the same\nAnswer:", "choices": ["date of birth", "shirt", "number of toenails", "hair length"], "gold": 2}, "request": {"context": "Answer:", "continuation": " hair length"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000054, "native_id": "1674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Answer:", "continuation": " commercial plane"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000055, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Answer:", "continuation": " private plane"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000055, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Answer:", "continuation": " bus"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000055, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "163", "query": "Question: The transportation with the most mass is likely a\nAnswer:", "choices": ["commercial plane", "private plane", "bus", "private car"], "gold": 0}, "request": {"context": "Answer:", "continuation": " private car"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000055, "native_id": "163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Answer:", "continuation": " meat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000056, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Answer:", "continuation": " compost"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000056, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Answer:", "continuation": " peas"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000056, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-49", "query": "Question: A rabbit may enjoy\nAnswer:", "choices": ["meat", "compost", "peas", "pebbles"], "gold": 2}, "request": {"context": "Answer:", "continuation": " pebbles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000056, "native_id": "7-49", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the deer are moved to a zoo"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000057, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the trail is expanded"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000057, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the mall has a nature park in it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000057, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-393", "query": "Question: A construction group wants to put a shopping center in town, but the only place available is a small nature park with a trail. Deer and other wildlife frequent the park, since it is the only place in the city where trees and fresh water are available for them. The construction group decides to build the shopping center, which means that\nAnswer:", "choices": ["the deer are moved to a zoo", "the trail is expanded", "the mall has a nature park in it", "the wildlife environment is destroyed"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the wildlife environment is destroyed"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000057, "native_id": "8-393", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 3pm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000058, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 2am"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000058, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 6pm"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000058, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "788", "query": "Question: Owls are likely to hunt at\nAnswer:", "choices": ["3pm", "2am", "6pm", "7am"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 7am"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000058, "native_id": "788", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the plants that were under the tree will have access to more light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000059, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the squirrels that were in that tree will have an easier time getting to their home"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000059, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Plants under the tree will get cooled off by the shade"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000059, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-29", "query": "Question: What could be a positive aspect of a tree being cut down?\nAnswer:", "choices": ["the plants that were under the tree will have access to more light", "the squirrels that were in that tree will have an easier time getting to their home", "Plants under the tree will get cooled off by the shade", "The sun will shine brighter than before"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The sun will shine brighter than before"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000059, "native_id": "9-29", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " grow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000060, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fertilize"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000060, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " reproduce"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000060, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-368", "query": "Question: Birds carrying away fruit helps the tree\nAnswer:", "choices": ["grow", "fertilize", "reproduce", "conquer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " conquer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000060, "native_id": "9-368", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the UFO is calling"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000061, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the UFO had been close"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000061, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the UFO is approaching"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000061, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-671", "query": "Question: If a UFO is flying overhead and looks small, then large, then\nAnswer:", "choices": ["the UFO is calling", "the UFO had been close", "the UFO is approaching", "the UFO is leaving"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the UFO is leaving"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000061, "native_id": "7-671", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " bears take a long winter sleep"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000062, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " dogs sit on command"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000062, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " seals clap for their trainers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000062, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1272", "query": "Question: Inherited behavior is exhibited when\nAnswer:", "choices": ["bears take a long winter sleep", "dogs sit on command", "seals clap for their trainers", "rats navigate thru a maze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " rats navigate thru a maze"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000062, "native_id": "1272", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Increased insect populations"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000063, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Clearing for farming"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000063, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " reduction in rainfall"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000063, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "648", "query": "Question: What likely explains deforestation?\nAnswer:", "choices": ["Increased insect populations", "Clearing for farming", "reduction in rainfall", "More carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " More carbon dioxide"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000063, "native_id": "648", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Answer:", "continuation": " steak"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000064, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Answer:", "continuation": " blood"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000064, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Answer:", "continuation": " nice weather"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000064, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1180", "query": "Question: Mosquitoes enjoy all the people at a BBQ in the summer for what reason?\nAnswer:", "choices": ["steak", "blood", "nice weather", "taking food"], "gold": 1}, "request": {"context": "Answer:", "continuation": " taking food"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000064, "native_id": "9-1180", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Answer:", "continuation": " dogs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000065, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000065, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Answer:", "continuation": " high peaks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000065, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-227", "query": "Question: The surface of the moon contains\nAnswer:", "choices": ["dogs", "water", "high peaks", "humans"], "gold": 2}, "request": {"context": "Answer:", "continuation": " humans"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000065, "native_id": "9-227", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " four boxes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000066, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " eight boxes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000066, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " two boxes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000066, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1582", "query": "Question: A tool used to identify the percent chance of a trait being passed down has at least\nAnswer:", "choices": ["four boxes", "eight boxes", "two boxes", "six boxes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " six boxes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000066, "native_id": "1582", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Answer:", "continuation": " asks for milk"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000067, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Answer:", "continuation": " asks for television"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000067, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Answer:", "continuation": " asks for water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000067, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-125", "query": "Question: A prisoner is kept in a stone room, unable to see the sun. The prisoner knows that he needs vitamin D to survive, so he\nAnswer:", "choices": ["asks for milk", "asks for television", "asks for water", "asks for sleep"], "gold": 0}, "request": {"context": "Answer:", "continuation": " asks for sleep"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000067, "native_id": "8-125", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Answer:", "continuation": " roots"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000068, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Answer:", "continuation": " tensile strength"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000068, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Answer:", "continuation": " plant temperature"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000068, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1923", "query": "Question: When trying to pull a rose out of the ground why do you encounter resistance?\nAnswer:", "choices": ["roots", "tensile strength", "plant temperature", "plant color"], "gold": 0}, "request": {"context": "Answer:", "continuation": " plant color"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000068, "native_id": "1923", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Answer:", "continuation": " rainy sky to fly in"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000069, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fast truck to drive"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000069, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a car with gasoline"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000069, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-229", "query": "Question: A bat flew through the sky without hitting anything due to which of these?\nAnswer:", "choices": ["rainy sky to fly in", "fast truck to drive", "a car with gasoline", "surfaces to reflect sound off"], "gold": 3}, "request": {"context": "Answer:", "continuation": " surfaces to reflect sound off"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000069, "native_id": "9-229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Answer:", "continuation": " puppies have soft fur"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000070, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Answer:", "continuation": " kittens look like their parents"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000070, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Answer:", "continuation": " all mammals are born with fur"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000070, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1702", "query": "Question: How can we see that the coloration of fur is an inherited characteristic?\nAnswer:", "choices": ["puppies have soft fur", "kittens look like their parents", "all mammals are born with fur", "baby rats are mostly bald"], "gold": 1}, "request": {"context": "Answer:", "continuation": " baby rats are mostly bald"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000070, "native_id": "1702", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Answer:", "continuation": " enables nuclear power to function"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000071, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Answer:", "continuation": " enables to emitting of light beams"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000071, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Answer:", "continuation": " enables gas powered motors to operate"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000071, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-260", "query": "Question: Decaying vegetation is part of the process that\nAnswer:", "choices": ["enables nuclear power to function", "enables to emitting of light beams", "enables gas powered motors to operate", "enables windmills to power electric grids"], "gold": 2}, "request": {"context": "Answer:", "continuation": " enables windmills to power electric grids"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000071, "native_id": "8-260", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Answer:", "continuation": " melted"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000072, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Answer:", "continuation": " dehydrated"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000072, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Answer:", "continuation": " bloated"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000072, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-491", "query": "Question: After a torrential downpour over a week, a man notices that the pond in his backyard is\nAnswer:", "choices": ["melted", "dehydrated", "bloated", "salted"], "gold": 2}, "request": {"context": "Answer:", "continuation": " salted"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000072, "native_id": "9-491", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " inverse"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000073, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " reverse"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000073, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " direct"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000073, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "75", "query": "Question: The amount of friction and the speed of an object have what kind of relationship?\nAnswer:", "choices": ["inverse", "reverse", "direct", "equal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " equal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000073, "native_id": "75", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " will turn into a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000074, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " will become bright green"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000074, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " will begin to recycle the nutrients that made up its structure"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000074, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1215", "query": "Question: A fallen leaf\nAnswer:", "choices": ["will turn into a tree", "will become bright green", "will begin to recycle the nutrients that made up its structure", "is likely to continue to grow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is likely to continue to grow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000074, "native_id": "1215", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The color of my hair"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000075, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The way I walk"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000075, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The size of a statue"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000075, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-93", "query": "Question: Over a period of time the weather can change\nAnswer:", "choices": ["The color of my hair", "The way I walk", "The size of a statue", "The sound a computer makes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The sound a computer makes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000075, "native_id": "8-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " an uptick in the number of leaves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000076, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a surge in leaf disease"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000076, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a gradual decrease in leaves"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000076, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-988", "query": "Question: Plant growth may cause\nAnswer:", "choices": ["an uptick in the number of leaves", "a surge in leaf disease", "a gradual decrease in leaves", "a rapid decline of the leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a rapid decline of the leaves"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000076, "native_id": "7-988", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " ice cube"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000077, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000077, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " electrical transformer"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000077, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1139", "query": "Question: The man's heart skipped a beat and he felt pain after touching which of these?\nAnswer:", "choices": ["ice cube", "water", "electrical transformer", "grass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " grass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000077, "native_id": "9-1139", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Oxygen"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000078, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Juice"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000078, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Wood"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000078, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1545", "query": "Question: Which substance is capable of dripping?\nAnswer:", "choices": ["Oxygen", "Juice", "Wood", "Lightning"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Lightning"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000078, "native_id": "1545", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it will be cooked perfectly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000079, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it will be bacteria laden"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000079, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it will become blackened"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000079, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-664", "query": "Question: If bacon is left too long on a hot stove top\nAnswer:", "choices": ["it will be cooked perfectly", "it will be bacteria laden", "it will become blackened", "it will be left raw"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it will be left raw"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000079, "native_id": "7-664", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Answer:", "continuation": " set to calories"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000080, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Answer:", "continuation": " set to volume"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000080, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Answer:", "continuation": " set to kilometers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000080, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-53", "query": "Question: the dashboard reading in a jaguar would likely be set to which of these?\nAnswer:", "choices": ["set to calories", "set to volume", "set to kilometers", "set to width"], "gold": 2}, "request": {"context": "Answer:", "continuation": " set to width"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000080, "native_id": "8-53", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Answer:", "continuation": " they could harm living things"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000081, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Answer:", "continuation": " they are very safe"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000081, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Answer:", "continuation": " they cause nothing serious"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000081, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1044", "query": "Question: are explosions safe?\nAnswer:", "choices": ["they could harm living things", "they are very safe", "they cause nothing serious", "none of these"], "gold": 0}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000081, "native_id": "7-1044", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the mountain pass"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000082, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the plain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000082, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the large hill"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000082, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1122", "query": "Question: The lowest temperature on the trip was at\nAnswer:", "choices": ["the mountain pass", "the plain", "the large hill", "the canyon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the canyon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000082, "native_id": "7-1122", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they are easier to make"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000083, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they let in less light"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000083, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they are cheaper to produce"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000083, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-79", "query": "Question: What is the benefit to using a frosted window film over a non treated windows?\nAnswer:", "choices": ["they are easier to make", "they let in less light", "they are cheaper to produce", "they are much stronger"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they are much stronger"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000083, "native_id": "9-79", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Answer:", "continuation": " copious amounts of seltzer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000084, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Answer:", "continuation": " scant amounts of sodium chloride"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000084, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Answer:", "continuation": " scant amounts of carbonation"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000084, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-157", "query": "Question: Ocean water contains\nAnswer:", "choices": ["copious amounts of seltzer", "scant amounts of sodium chloride", "scant amounts of carbonation", "copious amounts of the combination of Na and Cl"], "gold": 3}, "request": {"context": "Answer:", "continuation": " copious amounts of the combination of Na and Cl"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000084, "native_id": "7-157", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Answer:", "continuation": " blood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000085, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000085, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Answer:", "continuation": " prey"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000085, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1164", "query": "Question: A cheetah that runs all day will find it has lost a lot of\nAnswer:", "choices": ["blood", "water", "prey", "spots"], "gold": 1}, "request": {"context": "Answer:", "continuation": " spots"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000085, "native_id": "9-1164", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to give birth to live young"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000086, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to mate with it's partner"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000086, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to fly to warmer climates"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000086, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-63", "query": "Question: Beak shape can influence a bird's ability\nAnswer:", "choices": ["to give birth to live young", "to mate with it's partner", "to fly to warmer climates", "to chew up certain worms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to chew up certain worms"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000086, "native_id": "8-63", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Answer:", "continuation": " sensing shapes by light and shadows"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000087, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Answer:", "continuation": " having a unusually strong sense of smell"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000087, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Answer:", "continuation": " sensing nearby objects by temperature change"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000087, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-308", "query": "Question: Some blind people have demonstrated bat-like skills by:\nAnswer:", "choices": ["sensing shapes by light and shadows", "having a unusually strong sense of smell", "sensing nearby objects by temperature change", "using sound to 'see'"], "gold": 3}, "request": {"context": "Answer:", "continuation": " using sound to 'see'"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000087, "native_id": "8-308", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A session"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000088, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A Match"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000088, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A Season"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000088, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "326", "query": "Question: December 21st through March 20 is a three month period which is an example of what?\nAnswer:", "choices": ["A session", "A Match", "A Season", "Autumn"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Autumn"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000088, "native_id": "326", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Answer:", "continuation": " life sustaining elements"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000089, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Answer:", "continuation": " essentials for photosynthesis"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000089, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water and hydrogen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000089, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1184", "query": "Question: Seeds provide new plants with\nAnswer:", "choices": ["life sustaining elements", "essentials for photosynthesis", "water and hydrogen", "storage for roots"], "gold": 0}, "request": {"context": "Answer:", "continuation": " storage for roots"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000089, "native_id": "1184", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " ecru"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000090, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " red"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000090, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " black"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000090, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "359", "query": "Question: What is a more comfortable color to have for your automobile upholstery if living in a desert?\nAnswer:", "choices": ["ecru", "red", "black", "navy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " navy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000090, "native_id": "359", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fettuccine if left around"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000091, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " waxy leaves from certain plants"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000091, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " dead carcass meat from livestock"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000091, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-350", "query": "Question: The salamander could eat a large amounts of what?\nAnswer:", "choices": ["fettuccine if left around", "waxy leaves from certain plants", "dead carcass meat from livestock", "six legged winged organisms"], "gold": 3}, "request": {"context": "Answer:", "continuation": " six legged winged organisms"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000091, "native_id": "9-350", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a radio recording"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000092, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an emotion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000092, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a written message"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000092, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-140", "query": "Question: A person can see\nAnswer:", "choices": ["a radio recording", "an emotion", "a written message", "an abstract idea"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an abstract idea"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000092, "native_id": "7-140", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Answer:", "continuation": " malleable and nutritious"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000093, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Answer:", "continuation": " dry and sandy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000093, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Answer:", "continuation": " grainy and bitter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000093, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "591", "query": "Question: When looking for good soil for plants, typically what is optimal?\nAnswer:", "choices": ["malleable and nutritious", "dry and sandy", "grainy and bitter", "compact and hard"], "gold": 0}, "request": {"context": "Answer:", "continuation": " compact and hard"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000093, "native_id": "591", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Answer:", "continuation": " gold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000094, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Answer:", "continuation": " houses"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000094, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Answer:", "continuation": " feeders"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000094, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-391", "query": "Question: Animals are drawn to\nAnswer:", "choices": ["gold", "houses", "feeders", "Carbon Dioxide"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Carbon Dioxide"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000094, "native_id": "7-391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Answer:", "continuation": " looking for pretty leaves to collect"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000095, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Answer:", "continuation": " stockpiling rocks for fighting in the winter"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000095, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Answer:", "continuation": " stockpiling pecans for the frigid months"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000095, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1672", "query": "Question: Squirrels spend their fall\nAnswer:", "choices": ["looking for pretty leaves to collect", "stockpiling rocks for fighting in the winter", "stockpiling pecans for the frigid months", "collecting twigs to keep warm"], "gold": 2}, "request": {"context": "Answer:", "continuation": " collecting twigs to keep warm"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000095, "native_id": "1672", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A fire"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000096, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A tornado"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000096, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Rainfall"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000096, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-464", "query": "Question: Rainbows are always found after what?\nAnswer:", "choices": ["A fire", "A tornado", "Rainfall", "Cereal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Cereal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000096, "native_id": "9-464", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " government mentality"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000097, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " dirt quality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000097, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " town economies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000097, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-983", "query": "Question: Crop rotation has a positive impact on what?\nAnswer:", "choices": ["government mentality", "dirt quality", "town economies", "crop watering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " crop watering"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000097, "native_id": "9-983", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " rubbing it"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000098, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " seeing it"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000098, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " hearing it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000098, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-179", "query": "Question: the best method for detecting texture is\nAnswer:", "choices": ["rubbing it", "seeing it", "hearing it", "tasting it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " tasting it"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000098, "native_id": "9-179", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fast"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000099, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " large"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000099, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " hairless"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000099, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-942", "query": "Question: Cold-blooded animals are often\nAnswer:", "choices": ["fast", "large", "hairless", "slow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " slow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000099, "native_id": "7-942", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sunlight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Answer:", "continuation": " falling water molecules"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Answer:", "continuation": " blooming flowers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-100", "query": "Question: Grey clouds can bring\nAnswer:", "choices": ["sunlight", "falling water molecules", "blooming flowers", "drought conditions"], "gold": 1}, "request": {"context": "Answer:", "continuation": " drought conditions"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000100, "native_id": "7-100", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Answer:", "continuation": " ant"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Answer:", "continuation": " snake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Answer:", "continuation": " elephant"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-30", "query": "Question: Which animal is considered a predator?\nAnswer:", "choices": ["ant", "snake", "elephant", "giraffe"], "gold": 1}, "request": {"context": "Answer:", "continuation": " giraffe"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000101, "native_id": "9-30", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Answer:", "continuation": " enable plants to continue flourishing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Answer:", "continuation": " play an unimportant role in the reproduction process"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Answer:", "continuation": " are useless to plants"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1709", "query": "Question: Pollinators\nAnswer:", "choices": ["enable plants to continue flourishing", "play an unimportant role in the reproduction process", "are useless to plants", "are considered unwanted pests"], "gold": 0}, "request": {"context": "Answer:", "continuation": " are considered unwanted pests"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000102, "native_id": "1709", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Answer:", "continuation": " have their own dirt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Answer:", "continuation": " have their own sunlight"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Answer:", "continuation": " have a lot of sand"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-491", "query": "Question: A farmer harvests seeds from some plants, such as tomatoes, in order to plant them later on. These seeds, once planted\nAnswer:", "choices": ["have their own dirt", "have their own sunlight", "have a lot of sand", "contain their necessary nutrition"], "gold": 3}, "request": {"context": "Answer:", "continuation": " contain their necessary nutrition"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000103, "native_id": "8-491", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Answer:", "continuation": " genetics"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Answer:", "continuation": " skills"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Answer:", "continuation": " physical attributes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "44", "query": "Question: What type of characteristics are people not born with?\nAnswer:", "choices": ["genetics", "skills", "physical attributes", "height"], "gold": 1}, "request": {"context": "Answer:", "continuation": " height"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000104, "native_id": "44", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Answer:", "continuation": " steam"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Answer:", "continuation": " heat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1023", "query": "Question: A spinning object is used to make\nAnswer:", "choices": ["steam", "heat", "water", "electricity"], "gold": 3}, "request": {"context": "Answer:", "continuation": " electricity"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000105, "native_id": "1023", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " evaporation of the surrounding water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " discharge of liquid petroleum in the surrounding sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " improvement in the conditions of sea life"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1911", "query": "Question: One of the negative consequences of offshore oil platforms is\nAnswer:", "choices": ["evaporation of the surrounding water", "discharge of liquid petroleum in the surrounding sea", "improvement in the conditions of sea life", "increase in the birthrate of sea birds"], "gold": 1}, "request": {"context": "Answer:", "continuation": " increase in the birthrate of sea birds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000106, "native_id": "1911", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Answer:", "continuation": " 1000"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Answer:", "continuation": " 250"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Answer:", "continuation": " 5000"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "429", "query": "Question: The unit of measure derived from French word millilitre is a unit used for measuring volume generally used for values between 1 and\nAnswer:", "choices": ["1000", "250", "5000", "300"], "gold": 0}, "request": {"context": "Answer:", "continuation": " 300"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000107, "native_id": "429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Answer:", "continuation": " getting a new child"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Answer:", "continuation": " yelling at the mud"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Answer:", "continuation": " asking the child to stop"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-49", "query": "Question: A man's child runs through the yard in the sprinklers, getting mud all over their feet. The child then runs around on the porch, tracking mud everywhere. While the mud is still wet, the man decides to clean off the porch by\nAnswer:", "choices": ["getting a new child", "yelling at the mud", "asking the child to stop", "turning on the hose"], "gold": 3}, "request": {"context": "Answer:", "continuation": " turning on the hose"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000108, "native_id": "8-49", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " only happen in California"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cause solar and lunar eclipses"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " will break your vases"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "520", "query": "Question: Earthquakes\nAnswer:", "choices": ["only happen in California", "cause solar and lunar eclipses", "will break your vases", "make bridges much safer"], "gold": 2}, "request": {"context": "Answer:", "continuation": " make bridges much safer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000109, "native_id": "520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Answer:", "continuation": " how rough the footing will be"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Answer:", "continuation": " how bad the weather will be"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Answer:", "continuation": " how stable the ground will be"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1128", "query": "Question: A seismograph can accurately describe\nAnswer:", "choices": ["how rough the footing will be", "how bad the weather will be", "how stable the ground will be", "how shaky the horse will be"], "gold": 2}, "request": {"context": "Answer:", "continuation": " how shaky the horse will be"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000110, "native_id": "7-1128", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the sun is always bright"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the moon is brighter than stars"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the moon is brighter than a floodlight"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-394", "query": "Question: Light from further away may appear to be less bright than other, closer sources, such as in which instance?\nAnswer:", "choices": ["the sun is always bright", "the moon is brighter than stars", "the moon is brighter than a floodlight", "the sun is darker than the moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the sun is darker than the moon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000111, "native_id": "7-394", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Answer:", "continuation": " wither"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Answer:", "continuation": " persist"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Answer:", "continuation": " die"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1166", "query": "Question: A plant needs a specific climate to grow and\nAnswer:", "choices": ["wither", "persist", "die", "decay"], "gold": 1}, "request": {"context": "Answer:", "continuation": " decay"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000112, "native_id": "9-1166", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " music to be loud"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " music to be appealing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " reverberations to strike the eardrum"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-884", "query": "Question: Banging on a drum causes\nAnswer:", "choices": ["music to be loud", "music to be appealing", "reverberations to strike the eardrum", "concerts to sell out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " concerts to sell out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000113, "native_id": "7-884", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Mt. McKinley"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Lake Pontchartrain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The great lakes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-501", "query": "Question: What may have been formed by a volcano?\nAnswer:", "choices": ["Mt. McKinley", "Lake Pontchartrain", "The great lakes", "Niagara Falls"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Niagara Falls"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000114, "native_id": "9-501", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Answer:", "continuation": " old"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Answer:", "continuation": " dead"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Answer:", "continuation": " in an egg"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-757", "query": "Question: Humans, cats, dogs, and elephants are known as mammals because their kids are born alive. Non-mammalian babies are born\nAnswer:", "choices": ["old", "dead", "in an egg", "big"], "gold": 2}, "request": {"context": "Answer:", "continuation": " big"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000115, "native_id": "9-757", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " basements to flood due to weather"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " small birds to kill large birds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " waterfalls to flow backwards"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-725", "query": "Question: Wind can cause\nAnswer:", "choices": ["basements to flood due to weather", "small birds to kill large birds", "waterfalls to flow backwards", "stones to weather down to pebbles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " stones to weather down to pebbles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000116, "native_id": "7-725", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Answer:", "continuation": " include mice being able to navigate a maze"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Answer:", "continuation": " include learning to sit on command"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Answer:", "continuation": " include dolphins doing tricks for their trainers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1300", "query": "Question: Inherited characteristics\nAnswer:", "choices": ["include mice being able to navigate a maze", "include learning to sit on command", "include dolphins doing tricks for their trainers", "include spots on a ladybug"], "gold": 3}, "request": {"context": "Answer:", "continuation": " include spots on a ladybug"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000117, "native_id": "1300", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Measure the altitude of the bird using a reference point, such as a tall building."}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Identify the species of bird"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Ask the bird how high it was when it returns back to earth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-230", "query": "Question: How could we determine approximately how far a bird is from the ground?\nAnswer:", "choices": ["Measure the altitude of the bird using a reference point, such as a tall building.", "Identify the species of bird", "Ask the bird how high it was when it returns back to earth", "Measure the bird's mass"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Measure the bird's mass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000118, "native_id": "9-230", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Answer:", "continuation": " accidentally dropped into volcanoes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Answer:", "continuation": " confined to enclosures to be viewed by the public"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Answer:", "continuation": " found eating corn in the middle of North America"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-988", "query": "Question: Endangered pandas are sometimes\nAnswer:", "choices": ["accidentally dropped into volcanoes", "confined to enclosures to be viewed by the public", "found eating corn in the middle of North America", "made into delicious rare steaks"], "gold": 1}, "request": {"context": "Answer:", "continuation": " made into delicious rare steaks"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000119, "native_id": "9-988", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " reservoir"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " meat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " street"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-393", "query": "Question: Algae can be found in\nAnswer:", "choices": ["reservoir", "meat", "street", "tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " tree"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000120, "native_id": "9-393", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a campfire toasts bread"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a microwave heats soup"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a fire burns paper"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-823", "query": "Question: A toaster converts electrical energy into heat energy for toasting much like\nAnswer:", "choices": ["a campfire toasts bread", "a microwave heats soup", "a fire burns paper", "a small oven works"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a small oven works"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000121, "native_id": "7-823", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " mimicking other things"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " melting into sand"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " creating new homes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-24", "query": "Question: An octopus, when in danger and unable to swim to safety, may find itself\nAnswer:", "choices": ["mimicking other things", "melting into sand", "creating new homes", "mocking other fish"], "gold": 0}, "request": {"context": "Answer:", "continuation": " mocking other fish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000122, "native_id": "9-24", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Answer:", "continuation": " hit a toy baseball with a bat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Answer:", "continuation": " shake a baby rattle with your hand"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Answer:", "continuation": " bang the wall with your fist"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "570", "query": "Question: What does someone do when creating music?\nAnswer:", "choices": ["hit a toy baseball with a bat", "shake a baby rattle with your hand", "bang the wall with your fist", "pluck the strings of a fingerboard with your fingers"], "gold": 3}, "request": {"context": "Answer:", "continuation": " pluck the strings of a fingerboard with your fingers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000123, "native_id": "570", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Answer:", "continuation": " telescope views"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water level fluctuations"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Answer:", "continuation": " animal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-124", "query": "Question: The pull the human planet space rock orbiter has on certain bodies of dihydrogen monooxide results in?\nAnswer:", "choices": ["telescope views", "water level fluctuations", "animal", "plant harvesting"], "gold": 1}, "request": {"context": "Answer:", "continuation": " plant harvesting"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000124, "native_id": "9-124", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Answer:", "continuation": " baby oil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Answer:", "continuation": " corn syrup or"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Answer:", "continuation": " milk"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-199", "query": "Question: Since density = mass / volume, denser liquids such as water sink more than\nAnswer:", "choices": ["baby oil", "corn syrup or", "milk", "honey"], "gold": 0}, "request": {"context": "Answer:", "continuation": " honey"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000125, "native_id": "9-199", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a cabbage cell"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a bee cell"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a bear cell"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "767", "query": "Question: Photosynthesis can be performed by\nAnswer:", "choices": ["a cabbage cell", "a bee cell", "a bear cell", "a cat cell"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a cat cell"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000126, "native_id": "767", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Answer:", "continuation": " reverse"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Answer:", "continuation": " inverse"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Answer:", "continuation": " equal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "28", "query": "Question: The force exerted on an object and distance traveled have what kind of relationship?\nAnswer:", "choices": ["reverse", "inverse", "equal", "direct"], "gold": 3}, "request": {"context": "Answer:", "continuation": " direct"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000127, "native_id": "28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " photosynthesize"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " release waste"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " perform meiosis"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1134", "query": "Question: all cells use cellular respiration to\nAnswer:", "choices": ["photosynthesize", "release waste", "perform meiosis", "release energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " release energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000128, "native_id": "9-1134", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " reproductive purposes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " viewing sounds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sensing views"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1030", "query": "Question: The viewing oriented sensor of a prairie creature are for what?\nAnswer:", "choices": ["reproductive purposes", "viewing sounds", "sensing views", "sensing tastes"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sensing tastes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000129, "native_id": "9-1030", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Answer:", "continuation": " breaks for birds on the road"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Answer:", "continuation": " given a large breakfast"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Answer:", "continuation": " eats a few corn cobs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-18", "query": "Question: A dog is going to have to corral sheep for the afternoon, so it needs to prepare its body for the enormous workload ahead of it. The dog is\nAnswer:", "choices": ["breaks for birds on the road", "given a large breakfast", "eats a few corn cobs", "given two apples to watch"], "gold": 1}, "request": {"context": "Answer:", "continuation": " given two apples to watch"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000130, "native_id": "9-18", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the woman makes more money"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the woman makes the same amount of money"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the woman spends more money"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-378", "query": "Question: A woman sells bracelets that she makes. The bracelets gain popularity, and the woman makes incredibly large amounts of money from the sales. After a while, very few people are still buying the bracelets, so\nAnswer:", "choices": ["the woman makes more money", "the woman makes the same amount of money", "the woman spends more money", "the woman makes less money"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the woman makes less money"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000131, "native_id": "8-378", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Answer:", "continuation": " dry, grass covered, and humid"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Answer:", "continuation": " lush, green, and tropical"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Answer:", "continuation": " arid, parched, and sun-baked"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-677", "query": "Question: A desert environment is\nAnswer:", "choices": ["dry, grass covered, and humid", "lush, green, and tropical", "arid, parched, and sun-baked", "dry, damp, and lush"], "gold": 2}, "request": {"context": "Answer:", "continuation": " dry, damp, and lush"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000132, "native_id": "7-677", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Our Planet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The Atmosphere"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The Equator"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-786", "query": "Question: Seasons are caused by what rotating on its axis?\nAnswer:", "choices": ["Our Planet", "The Atmosphere", "The Equator", "The Sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The Sun"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000133, "native_id": "9-786", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Answer:", "continuation": " tile flooring"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Answer:", "continuation": " human flesh"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Answer:", "continuation": " hockey stick"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-463", "query": "Question: Which is best an letting electricity pass through?\nAnswer:", "choices": ["tile flooring", "human flesh", "hockey stick", "a steak knife"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a steak knife"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000134, "native_id": "9-463", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " large, bowl-shaped cavities in the ground"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " aliens and foreign foods"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " small dents in the planet's core"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-71", "query": "Question: Asteroids crashing on planets can leave behind\nAnswer:", "choices": ["large, bowl-shaped cavities in the ground", "aliens and foreign foods", "small dents in the planet's core", "lakes filled with salty water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " lakes filled with salty water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000135, "native_id": "7-71", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " NH4"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " C4H4"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " CO2"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1053", "query": "Question: What is the formula of the substance which best helps plants grow\nAnswer:", "choices": ["NH4", "C4H4", "CO2", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " H2O"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000136, "native_id": "9-1053", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Yelling really loud"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Soaring the skies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Going walking"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-437", "query": "Question: You can experience a change of pressure when\nAnswer:", "choices": ["Yelling really loud", "Soaring the skies", "Going walking", "riding a bike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " riding a bike"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000137, "native_id": "9-437", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the creek water getting colder"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a parched creek bed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " tributaries branching off from the creek"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1787", "query": "Question: A small creek absorbing heat energy can result in\nAnswer:", "choices": ["the creek water getting colder", "a parched creek bed", "tributaries branching off from the creek", "a runoff of extra water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a runoff of extra water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000138, "native_id": "1787", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Australia"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " in Canada"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " United States"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-107", "query": "Question: Summertime  happens during June in all but which location?\nAnswer:", "choices": ["Australia", "in Canada", "United States", "Europe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Europe"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000139, "native_id": "7-107", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Answer:", "continuation": " lake"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Answer:", "continuation": " pool"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Answer:", "continuation": " stomach"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "769", "query": "Question: It takes more water to fill a bathtub than a\nAnswer:", "choices": ["lake", "pool", "stomach", "holding tank"], "gold": 2}, "request": {"context": "Answer:", "continuation": " holding tank"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000140, "native_id": "769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Brazil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Indiana"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Answer:", "continuation": " London"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-73", "query": "Question: The eighth month of the year is winter in\nAnswer:", "choices": ["Brazil", "Indiana", "London", "Canada"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Canada"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000141, "native_id": "9-73", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " dogs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a tan"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a ball"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1194", "query": "Question: birds use their peckers to catch\nAnswer:", "choices": ["dogs", "a tan", "a ball", "bees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " bees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000142, "native_id": "9-1194", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fuel costs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the economy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Answer:", "continuation": " housing value"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-416", "query": "Question: transplanting seedling oaks has a positive impact on\nAnswer:", "choices": ["fuel costs", "the economy", "housing value", "the environment"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the environment"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000143, "native_id": "9-416", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " humans throwing a spear through an animal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " humans chewing on boiled animal muscles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " humans gathering animals in a gate"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "470", "query": "Question: What is an example of hunting?\nAnswer:", "choices": ["humans throwing a spear through an animal", "humans chewing on boiled animal muscles", "humans gathering animals in a gate", "humans plucking fruit from a tree"], "gold": 0}, "request": {"context": "Answer:", "continuation": " humans plucking fruit from a tree"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000144, "native_id": "470", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " paper"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " wood"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " stone"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1297", "query": "Question: Which of these has shape that changes depending on the container which it resides within?\nAnswer:", "choices": ["paper", "wood", "stone", "orange juice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " orange juice"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000145, "native_id": "1297", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Sweet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Salty"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Answer:", "continuation": " bitter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-346", "query": "Question: What would be the flavor if you ate the item that fell and is thought to have hit Sir Issac Newton's head\nAnswer:", "choices": ["Sweet", "Salty", "bitter", "sour"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sour"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000146, "native_id": "8-346", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " stone"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " bacteria"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-807", "query": "Question: Earth's four layers are comprised mainly of\nAnswer:", "choices": ["stone", "bacteria", "water", "air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000147, "native_id": "7-807", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the freezing point"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the prime point"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the boiling point"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-463", "query": "Question: if coffee sits in the fridge and loses its liquid form, what is that point known as?\nAnswer:", "choices": ["the freezing point", "the prime point", "the boiling point", "the melting point"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the melting point"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000148, "native_id": "8-463", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Answer:", "continuation": " children"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Answer:", "continuation": " birds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Answer:", "continuation": " fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-110", "query": "Question: Mammals give birth to live\nAnswer:", "choices": ["children", "birds", "fish", "insects"], "gold": 0}, "request": {"context": "Answer:", "continuation": " insects"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000149, "native_id": "9-110", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it loses its magnetization"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it loses its charge"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it may become magnetized"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1611", "query": "Question: If a battery in an electromagnet is active, then what will happen to a nail in that electromagnet?\nAnswer:", "choices": ["it loses its magnetization", "it loses its charge", "it may become magnetized", "it gains a charge"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it gains a charge"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000150, "native_id": "1611", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " luck"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " humans"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " gold"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-942", "query": "Question: Which of these is a factor in the shape of a fern's seed?\nAnswer:", "choices": ["luck", "humans", "gold", "inheritance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " inheritance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000151, "native_id": "9-942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " An Elephant"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A school notebook"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A boat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1102", "query": "Question: Which is recyclable?\nAnswer:", "choices": ["An Elephant", "A school notebook", "A boat", "A lake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A lake"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000152, "native_id": "9-1102", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it is a place to emigrate from"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it is an ideal habitat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it is an unsustainable habitat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-774", "query": "Question: if the population in a habitat is on a steady decline, what condition is the habitat?\nAnswer:", "choices": ["it is a place to emigrate from", "it is an ideal habitat", "it is an unsustainable habitat", "it is a thriving abode"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it is a thriving abode"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000153, "native_id": "9-774", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water and soda"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water and oil"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sand and rocks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-333", "query": "Question: A teacher wants to show how to combine two substances together. The two things that he can use in order to mix them completely are\nAnswer:", "choices": ["water and soda", "water and oil", "sand and rocks", "salt and bark"], "gold": 0}, "request": {"context": "Answer:", "continuation": " salt and bark"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000154, "native_id": "8-333", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Answer:", "continuation": " spider bites"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Answer:", "continuation": " vomiting"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Answer:", "continuation": " placebo effect"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-573", "query": "Question: Medicine is used to cure but can cause people to have allergic reactions such as\nAnswer:", "choices": ["spider bites", "vomiting", "placebo effect", "dance fever"], "gold": 1}, "request": {"context": "Answer:", "continuation": " dance fever"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000155, "native_id": "9-573", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Answer:", "continuation": " damage to local foliage"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Answer:", "continuation": " rainbows across the sky"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Answer:", "continuation": " growth of local flora"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1955", "query": "Question: Lightning may lead to\nAnswer:", "choices": ["damage to local foliage", "rainbows across the sky", "growth of local flora", "firefighters getting the night off"], "gold": 0}, "request": {"context": "Answer:", "continuation": " firefighters getting the night off"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000156, "native_id": "1955", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Answer:", "continuation": " 5 p.m."}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Answer:", "continuation": " 12 p.m."}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Answer:", "continuation": " 3 a.m."}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-45", "query": "Question: When would a nocturnal predator most likely hunt?\nAnswer:", "choices": ["5 p.m.", "12 p.m.", "3 a.m.", "10 a.m."], "gold": 2}, "request": {"context": "Answer:", "continuation": " 10 a.m."}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000157, "native_id": "8-45", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Answer:", "continuation": " inside a disc golf driver"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Answer:", "continuation": " inside of a brass pipe"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a mile up in the sky"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-674", "query": "Question: Where water be located in its gas form?\nAnswer:", "choices": ["inside a disc golf driver", "inside of a brass pipe", "a mile up in the sky", "inside a leather baseball"], "gold": 2}, "request": {"context": "Answer:", "continuation": " inside a leather baseball"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000158, "native_id": "9-674", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " carbon dioxide"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " light"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "898", "query": "Question: Green parts of a life form absorb\nAnswer:", "choices": ["carbon dioxide", "light", "oxygen", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000159, "native_id": "898", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the new species will become herbivores"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " prey will experience an increase in population"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " prey will experience a drop in population"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1159", "query": "Question: If a new species of predator joins a community\nAnswer:", "choices": ["the new species will become herbivores", "prey will experience an increase in population", "prey will experience a drop in population", "the old species will die out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the old species will die out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000160, "native_id": "7-1159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " It's dead"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " For water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " For food"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "568", "query": "Question: Why would a perennial plant with an elongated stem a frequently used for lumber fall to the ground?\nAnswer:", "choices": ["It's dead", "For water", "For food", "For sun"], "gold": 0}, "request": {"context": "Answer:", "continuation": " For sun"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000161, "native_id": "568", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Answer:", "continuation": " weasel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Answer:", "continuation": " snow fox"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Answer:", "continuation": " wolf"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-877", "query": "Question: I'm an animal with a white fur and a large fluffy tail that lives in arctic regions; what am I?\nAnswer:", "choices": ["weasel", "snow fox", "wolf", "polar bear"], "gold": 1}, "request": {"context": "Answer:", "continuation": " polar bear"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000162, "native_id": "9-877", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a vitamin that prevents blood loss"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a vitamin that treats amino acid deficiency"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a group of fat-soluble secosteroids"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "406", "query": "Question: Dairy is a source of\nAnswer:", "choices": ["a vitamin that prevents blood loss", "a vitamin that treats amino acid deficiency", "a group of fat-soluble secosteroids", "a vitamin that helps treat liver problems"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a vitamin that helps treat liver problems"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000163, "native_id": "406", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Answer:", "continuation": " magnetized"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Na"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Answer:", "continuation": " compass"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1132", "query": "Question: Atomic 26 is drawn to a device, it could be\nAnswer:", "choices": ["magnetized", "Na", "compass", "K"], "gold": 0}, "request": {"context": "Answer:", "continuation": " K"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000164, "native_id": "7-1132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " tadpoles change to different animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " tadpoles are really just fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they are young frogs still growing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-479", "query": "Question: why do tadpoles change into frogs?\nAnswer:", "choices": ["tadpoles change to different animals", "tadpoles are really just fish", "they are young frogs still growing", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000165, "native_id": "7-479", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Chickpeas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Chocolate"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Steak"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "609", "query": "Question: What do cows eat?\nAnswer:", "choices": ["Chickpeas", "Chocolate", "Steak", "Poultry"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Poultry"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000166, "native_id": "609", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Window"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Rubik's Cube"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Ship Anchor"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1568", "query": "Question: Which object conducts electricity?\nAnswer:", "choices": ["Window", "Rubik's Cube", "Ship Anchor", "Boulder"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Boulder"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000167, "native_id": "1568", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Answer:", "continuation": " from roots to leaves of a daffodil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Answer:", "continuation": " from a rose's leaves to the atmosphere"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Answer:", "continuation": " from a daisy's leaves into it's underground support system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-418", "query": "Question: Glucose travels\nAnswer:", "choices": ["from roots to leaves of a daffodil", "from a rose's leaves to the atmosphere", "from a daisy's leaves into it's underground support system", "from the sun to a sunflower's buds"], "gold": 2}, "request": {"context": "Answer:", "continuation": " from the sun to a sunflower's buds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000168, "native_id": "9-418", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Answer:", "continuation": " only happens in the summer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Answer:", "continuation": " is like nature's disappearing water trick"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Answer:", "continuation": " is caused by snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1050", "query": "Question: Evaporation\nAnswer:", "choices": ["only happens in the summer", "is like nature's disappearing water trick", "is caused by snow", "involves the disappearance of sunlight"], "gold": 1}, "request": {"context": "Answer:", "continuation": " involves the disappearance of sunlight"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000169, "native_id": "7-1050", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Corn"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Runoff"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-510", "query": "Question: In order for crops to grow food safely, pesticides are used on them. When it floods, this causes t he what to be poisonous?\nAnswer:", "choices": ["air", "Corn", "Runoff", "farmers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " farmers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000170, "native_id": "9-510", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Answer:", "continuation": " friction"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Answer:", "continuation": " metal"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Answer:", "continuation": " wishing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-519", "query": "Question: The boy was able to warm the fireplace without a lighter thanks to what?\nAnswer:", "choices": ["friction", "metal", "wishing", "magic"], "gold": 0}, "request": {"context": "Answer:", "continuation": " magic"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000171, "native_id": "9-519", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the Sahara"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Antarctica"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the Appalachian mountains"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-637", "query": "Question: Where would a duck like to live?\nAnswer:", "choices": ["the Sahara", "Antarctica", "the Appalachian mountains", "Death Valley"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Death Valley"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000172, "native_id": "9-637", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Atlantic menhaden"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Swedish fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Answer:", "continuation": " gummy fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "473", "query": "Question: What do tuna eat?\nAnswer:", "choices": ["Atlantic menhaden", "Swedish fish", "gummy fish", "laminariales"], "gold": 0}, "request": {"context": "Answer:", "continuation": " laminariales"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000173, "native_id": "473", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Answer:", "continuation": " flowers blooming"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Answer:", "continuation": " grass turning brown"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Answer:", "continuation": " trees growing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-445", "query": "Question: A woman notices that she is depressed every autumn, and wonders why. A friend suggests to her that perhaps certain changes that take place as seasons move from warm to cold may be having an effect on her. When pressed for an example of these changes, the friend cites\nAnswer:", "choices": ["flowers blooming", "grass turning brown", "trees growing", "blossoms blooming"], "gold": 1}, "request": {"context": "Answer:", "continuation": " blossoms blooming"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000174, "native_id": "8-445", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " it has never happened"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " yes it is standard"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " it is abnormal and weird"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-575", "query": "Question: is it normal for an adult animal to lay eggs?\nAnswer:", "choices": ["it has never happened", "yes it is standard", "it is abnormal and weird", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000175, "native_id": "9-575", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " add more fat to their diet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " cut back their caloric intake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " let them sleep more"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-284", "query": "Question: If your dog is overweight\nAnswer:", "choices": ["add more fat to their diet", "cut back their caloric intake", "let them sleep more", "increase their caloric intake"], "gold": 1}, "request": {"context": "Answer:", "continuation": " increase their caloric intake"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000176, "native_id": "7-284", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " to detect when a traffic light changes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " detect sour flavors in candy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " hear music at concerts"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-135", "query": "Question: Eyes allow humans\nAnswer:", "choices": ["to detect when a traffic light changes", "detect sour flavors in candy", "hear music at concerts", "detect acrid odors in the air"], "gold": 0}, "request": {"context": "Answer:", "continuation": " detect acrid odors in the air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000177, "native_id": "8-135", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " become fossils over night"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " may end up reanimated over time"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " develop characteristics for survival"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "397", "query": "Question: Organisms covered by layers of sediment\nAnswer:", "choices": ["become fossils over night", "may end up reanimated over time", "develop characteristics for survival", "may end up fueling a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " may end up fueling a car"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000178, "native_id": "397", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Answer:", "continuation": " counties"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Answer:", "continuation": " north of equator"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Answer:", "continuation": " states"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-32", "query": "Question: The winter solstice is on December 21st in the\nAnswer:", "choices": ["counties", "north of equator", "states", "southern hemisphere"], "gold": 1}, "request": {"context": "Answer:", "continuation": " southern hemisphere"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000179, "native_id": "9-32", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Answer:", "continuation": " bleaching"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Answer:", "continuation": " burning"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fossilization"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "48", "query": "Question: A lake environment is a good setup for what to happen to organic remains?\nAnswer:", "choices": ["bleaching", "burning", "fossilization", "drying"], "gold": 2}, "request": {"context": "Answer:", "continuation": " drying"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000180, "native_id": "48", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " this was only created by humans"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " humans predate fossil fuel formation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " significant supplies accumulated prior"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-69", "query": "Question: were there fossil fuels in the ground when humans evolved?\nAnswer:", "choices": ["this was only created by humans", "humans predate fossil fuel formation", "significant supplies accumulated prior", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000181, "native_id": "8-69", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Answer:", "continuation": " some water is around it"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Answer:", "continuation": " water rained on it"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Answer:", "continuation": " natural waters weathered it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-159", "query": "Question: The Grand Canyon is massive, with large, high peaks and very deep lows, which was formed when\nAnswer:", "choices": ["some water is around it", "water rained on it", "natural waters weathered it", "a pool was opened"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a pool was opened"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000182, "native_id": "9-159", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " bananas"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " electricity"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-317", "query": "Question: What type of useful product can be made from the moving winds?\nAnswer:", "choices": ["wood", "bananas", "electricity", "metal"], "gold": 2}, "request": {"context": "Answer:", "continuation": " metal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000183, "native_id": "9-317", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Answer:", "continuation": " metal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Answer:", "continuation": " rivers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "423", "query": "Question: The sides of the canyon are\nAnswer:", "choices": ["metal", "water", "rivers", "stone"], "gold": 3}, "request": {"context": "Answer:", "continuation": " stone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000184, "native_id": "423", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a man who lives next to a landfill"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a man who lives in a city with the best air quality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-304", "query": "Question: which of these people would have the worst air quality at their residence?\nAnswer:", "choices": ["a man who lives next to a landfill", "a man who lives in a city with the best air quality", "none of these", "a man who lives in a great suburb"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a man who lives in a great suburb"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000185, "native_id": "8-304", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to rewind 5 minutes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to tell what will happen 5 minutes from now"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to voice the time"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "785", "query": "Question: What is a stopwatch used for?\nAnswer:", "choices": ["to rewind 5 minutes", "to tell what will happen 5 minutes from now", "to voice the time", "to measure minutes and hours"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to measure minutes and hours"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000186, "native_id": "785", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Answer:", "continuation": " stay the same"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fluctuate wildly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Answer:", "continuation": " decrease"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-1087", "query": "Question: As a drought worsens the level at an aquifer will\nAnswer:", "choices": ["stay the same", "fluctuate wildly", "decrease", "increase"], "gold": 2}, "request": {"context": "Answer:", "continuation": " increase"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000187, "native_id": "9-1087", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Answer:", "continuation": " marble statues"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Answer:", "continuation": " molten magma"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Answer:", "continuation": " subterranean termites"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "485", "query": "Question: Wind frequently helps transport from one place to another\nAnswer:", "choices": ["marble statues", "molten magma", "subterranean termites", "exposed topsoil"], "gold": 3}, "request": {"context": "Answer:", "continuation": " exposed topsoil"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000188, "native_id": "485", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " bike helmet"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " avoiding accidents"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " friction"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-908", "query": "Question: When a kid slams on the brakes on their bike what is caused?\nAnswer:", "choices": ["bike helmet", "avoiding accidents", "friction", "gearing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " gearing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000189, "native_id": "9-908", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " crystals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " feathers"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cell walls"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1231", "query": "Question: Sources of spices have\nAnswer:", "choices": ["crystals", "feathers", "cell walls", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " craters"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000190, "native_id": "1231", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Answer:", "continuation": " leads to longer drought of resources"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Answer:", "continuation": " leads to longer availability of resources"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Answer:", "continuation": " leads to more consumption"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "810", "query": "Question: Conservation\nAnswer:", "choices": ["leads to longer drought of resources", "leads to longer availability of resources", "leads to more consumption", "leads to short supply of resources"], "gold": 1}, "request": {"context": "Answer:", "continuation": " leads to short supply of resources"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000191, "native_id": "810", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " squirrel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " woodpecker"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " monkey"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "158", "query": "Question: A tree is not the habitat of a\nAnswer:", "choices": ["squirrel", "woodpecker", "monkey", "lion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " lion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000192, "native_id": "158", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a change in the direction of a stream"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a change in ocean temperatures"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Answer:", "continuation": " an increase in rainy weather"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-445", "query": "Question: Erosion could lead to\nAnswer:", "choices": ["a change in the direction of a stream", "a change in ocean temperatures", "an increase in rainy weather", "an increase in plants and animals"], "gold": 0}, "request": {"context": "Answer:", "continuation": " an increase in plants and animals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000193, "native_id": "7-445", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it's tilted towards Jupiter"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it's angled towards the moon"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it's angled towards the largest star in the solar system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1502", "query": "Question: A hemisphere experiences summer when\nAnswer:", "choices": ["it's tilted towards Jupiter", "it's angled towards the moon", "it's angled towards the largest star in the solar system", "it spins counter clockwise on Earth's axis"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it spins counter clockwise on Earth's axis"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000194, "native_id": "1502", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Jul 4th"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Feb 29th"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Answer:", "continuation": " May 3rd"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1200", "query": "Question: What date is the amount of daylight minimized\nAnswer:", "choices": ["Jul 4th", "Feb 29th", "May 3rd", "Sep 1st"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Sep 1st"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000195, "native_id": "1200", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Eucerin pH5 range"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Sagittal plane"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Answer:", "continuation": " pyogenic vibrio"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "437", "query": "Question: Which term is involved with protection by skin?\nAnswer:", "choices": ["Eucerin pH5 range", "Sagittal plane", "pyogenic vibrio", "popliteus"], "gold": 0}, "request": {"context": "Answer:", "continuation": " popliteus"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000196, "native_id": "437", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " It is made from rock"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " It eats three meals a day"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " It has a loving family"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-205", "query": "Question: The reason Earth is so sturdy is because\nAnswer:", "choices": ["It is made from rock", "It eats three meals a day", "It has a loving family", "It is made from metal"], "gold": 0}, "request": {"context": "Answer:", "continuation": " It is made from metal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000197, "native_id": "8-205", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Answer:", "continuation": " produces fruit"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Answer:", "continuation": " grows faster"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fails to grow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-270", "query": "Question: A plant left in the dark\nAnswer:", "choices": ["produces fruit", "grows faster", "fails to grow", "gets greener"], "gold": 2}, "request": {"context": "Answer:", "continuation": " gets greener"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000198, "native_id": "9-270", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " heat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " metal"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " lithium-ion"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-130", "query": "Question: A boy wants to use his Walkman so that he can listen to some music. When he tries to turn it on, it us unable to, and the boy realizes that he will need\nAnswer:", "choices": ["heat", "metal", "lithium-ion", "plastic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " plastic"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000199, "native_id": "8-130", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " axial planetary rotation"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " comets"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " planetary formation"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "229", "query": "Question: Nuclear activity is the cause of what celestial occurrence?\nAnswer:", "choices": ["axial planetary rotation", "comets", "planetary formation", "the sun's rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the sun's rays"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000200, "native_id": "229", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Answer:", "continuation": " River"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Ocean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-390", "query": "Question: Which source provides the safest water?\nAnswer:", "choices": ["River", "Sea", "Ocean", "Rain"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000201, "native_id": "9-390", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Answer:", "continuation": " spread out some fur"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Answer:", "continuation": " has black on his ears"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Answer:", "continuation": " passed down inherited characteristics"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-107", "query": "Question: A rabbit has a litter of bunnies! Most of the babies are white, just like the mother rabbit, but one baby has brown spots, like the father rabbit. The father rabbit\nAnswer:", "choices": ["spread out some fur", "has black on his ears", "passed down inherited characteristics", "is the same size as the mother"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is the same size as the mother"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000202, "native_id": "8-107", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Answer:", "continuation": " convert sunlight to sand"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Answer:", "continuation": " provide food sources for others"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Answer:", "continuation": " be producers in an ecosystem"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-527", "query": "Question: Photosynthesis means plants are unable to\nAnswer:", "choices": ["convert sunlight to sand", "provide food sources for others", "be producers in an ecosystem", "make their own food"], "gold": 0}, "request": {"context": "Answer:", "continuation": " make their own food"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000203, "native_id": "7-527", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " rabbits are born with ears"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " there was a lot of food"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " genetic contributors had long ears"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-333", "query": "Question: Through DNA, a rabbit will have long ears if\nAnswer:", "choices": ["rabbits are born with ears", "there was a lot of food", "genetic contributors had long ears", "parents were also rabbits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " parents were also rabbits"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000204, "native_id": "7-333", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " giant fish"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " enormous crabs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " whitetails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-44", "query": "Question: There are various creatures that live in forests, such as\nAnswer:", "choices": ["giant fish", "enormous crabs", "whitetails", "desert jackals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " desert jackals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000205, "native_id": "9-44", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Answer:", "continuation": " water splashing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Answer:", "continuation": " running fast"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Answer:", "continuation": " long hands"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-160", "query": "Question: An octopus protects itself with\nAnswer:", "choices": ["water splashing", "running fast", "long hands", "pigment squirting"], "gold": 3}, "request": {"context": "Answer:", "continuation": " pigment squirting"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000206, "native_id": "7-160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Answer:", "continuation": " induces organisms to use more of their resources"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Answer:", "continuation": " causes an increase in use of resources"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Answer:", "continuation": " causes an uptick in birthrate"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1942", "query": "Question: Resources decreasing in an environment\nAnswer:", "choices": ["induces organisms to use more of their resources", "causes an increase in use of resources", "causes an uptick in birthrate", "induces organisms to be more economical with resources"], "gold": 3}, "request": {"context": "Answer:", "continuation": " induces organisms to be more economical with resources"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000207, "native_id": "1942", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " insects"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " weathering"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " lava"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-597", "query": "Question: The sidewalk next to a house having a crack in it and having vegetation growing from it is considered?\nAnswer:", "choices": ["insects", "weathering", "lava", "erosion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " erosion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000208, "native_id": "9-597", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Answer:", "continuation": " millions of years"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Answer:", "continuation": " 1 day"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Answer:", "continuation": " 10 days"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-35", "query": "Question: If you wanted to make a necklace, how long would you have to wait for the materials to appear inside the Earth?\nAnswer:", "choices": ["millions of years", "1 day", "10 days", "100 days"], "gold": 0}, "request": {"context": "Answer:", "continuation": " 100 days"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000209, "native_id": "9-35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Answer:", "continuation": " change the moon into cheese"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Answer:", "continuation": " alter the way the moon's facade looks"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Answer:", "continuation": " change moon lakes into vapor"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1161", "query": "Question: Moon phases\nAnswer:", "choices": ["change the moon into cheese", "alter the way the moon's facade looks", "change moon lakes into vapor", "cause lunar eclipse every day"], "gold": 1}, "request": {"context": "Answer:", "continuation": " cause lunar eclipse every day"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000210, "native_id": "1161", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they would be scalded"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Answer:", "continuation": " organs would remain uneffected"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they would begin to decay"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-171", "query": "Question: If hot water were poured on an arm, what would happen to internal organs?\nAnswer:", "choices": ["they would be scalded", "organs would remain uneffected", "they would begin to decay", "they would experience pain"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they would experience pain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000211, "native_id": "7-171", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Answer:", "continuation": " eat organic chicken"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Answer:", "continuation": " eat eggs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Answer:", "continuation": " eat beef"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1139", "query": "Question: Barnyard bovines\nAnswer:", "choices": ["eat organic chicken", "eat eggs", "eat beef", "eat alfalfa hay"], "gold": 3}, "request": {"context": "Answer:", "continuation": " eat alfalfa hay"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000212, "native_id": "1139", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " tropical plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " tons of sun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " massive rain totals"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1924", "query": "Question: Desert environments features\nAnswer:", "choices": ["tropical plants", "tons of sun", "massive rain totals", "icy precipitation"], "gold": 1}, "request": {"context": "Answer:", "continuation": " icy precipitation"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000213, "native_id": "1924", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Answer:", "continuation": " army of ants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a community"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a toy store"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-440", "query": "Question: a large cluster of humans, dogs, apple trees, atmosphere and more can be called\nAnswer:", "choices": ["army of ants", "a community", "a toy store", "a shopping mall"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a shopping mall"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000214, "native_id": "9-440", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the lungs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the kidney"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the heart"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-528", "query": "Question: A person is considering various organs, and is looking at which ones will be most muscular. A contender for most muscular is\nAnswer:", "choices": ["the lungs", "the kidney", "the heart", "the liver"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the liver"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000215, "native_id": "9-528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Answer:", "continuation": " rabbit and hare"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Answer:", "continuation": " mule and hinny"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cat and catfish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "170", "query": "Question: Which pair don't reproduce the same way?\nAnswer:", "choices": ["rabbit and hare", "mule and hinny", "cat and catfish", "caterpillar and butterfly"], "gold": 2}, "request": {"context": "Answer:", "continuation": " caterpillar and butterfly"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000216, "native_id": "170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Answer:", "continuation": " toad"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Answer:", "continuation": " snake"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Answer:", "continuation": " turtle"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "395", "query": "Question: Which of the following is warm blooded?\nAnswer:", "choices": ["toad", "snake", "turtle", "skunk"], "gold": 3}, "request": {"context": "Answer:", "continuation": " skunk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000217, "native_id": "395", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a manager"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an agent"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a meal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-633", "query": "Question: A male bird spots a female of his species and begins a fancy dance, flashing his bright feathers around in the air, showing off. This male is attempting to procure\nAnswer:", "choices": ["a manager", "an agent", "a meal", "a reproductive companion"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a reproductive companion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000218, "native_id": "9-633", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a quarry"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " concerts"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " forests"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-504", "query": "Question: Vast quantities of metal can be obtained from\nAnswer:", "choices": ["a quarry", "concerts", "forests", "salt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " salt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000219, "native_id": "9-504", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Heat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Kinetic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Magnetic"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-192", "query": "Question: Roasting a turkey requires adding what type of energy\nAnswer:", "choices": ["Heat", "Kinetic", "Magnetic", "Chemical"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Chemical"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000220, "native_id": "8-192", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Answer:", "continuation": " liberally use water for everything"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Answer:", "continuation": " allow plants to flourish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Answer:", "continuation": " require people to move"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1108", "query": "Question: Dry environments often\nAnswer:", "choices": ["liberally use water for everything", "allow plants to flourish", "require people to move", "institute rules about water usage"], "gold": 3}, "request": {"context": "Answer:", "continuation": " institute rules about water usage"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000221, "native_id": "7-1108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Answer:", "continuation": " blood moves in a body"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Answer:", "continuation": " leaves move in the wind"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water moves in a system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-852", "query": "Question: Phloem moves things around a plant similar to how\nAnswer:", "choices": ["blood moves in a body", "leaves move in the wind", "water moves in a system", "cars move on a street"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cars move on a street"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000222, "native_id": "7-852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Arizona"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Georgia"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Florida"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "761", "query": "Question: Where would a polar bear be most comfortable?\nAnswer:", "choices": ["Arizona", "Georgia", "Florida", "Nebraska"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Nebraska"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000223, "native_id": "761", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Answer:", "continuation": " light is far away"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Answer:", "continuation": " light is more easily seen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Answer:", "continuation": " light is more distant"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-318", "query": "Question: When a plane is in the sky and is several miles away, the light seen is barely visible, but when it is drawing near\nAnswer:", "choices": ["light is far away", "light is more easily seen", "light is more distant", "light is further away"], "gold": 1}, "request": {"context": "Answer:", "continuation": " light is further away"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000224, "native_id": "8-318", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Answer:", "continuation": " flutes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sandals"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Answer:", "continuation": " earplugs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "636", "query": "Question: Members of rock bands often perform with\nAnswer:", "choices": ["flutes", "sandals", "earplugs", "gloves"], "gold": 2}, "request": {"context": "Answer:", "continuation": " gloves"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000225, "native_id": "636", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Answer:", "continuation": " arms are resting"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Answer:", "continuation": " hair is growing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Answer:", "continuation": " smiles are invisible"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-444", "query": "Question: Muscles move bones to produce movement like when\nAnswer:", "choices": ["arms are resting", "hair is growing", "smiles are invisible", "toes are wiggled"], "gold": 3}, "request": {"context": "Answer:", "continuation": " toes are wiggled"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000226, "native_id": "7-444", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " decaying organic material from sealife"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " energy is accessed underwater from tides"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " drills to access oil supplies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-57", "query": "Question: How is electricity produced from the ocean?\nAnswer:", "choices": ["decaying organic material from sealife", "energy is accessed underwater from tides", "drills to access oil supplies", "chemical reactions produced from the salt in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " chemical reactions produced from the salt in the water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000227, "native_id": "8-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Answer:", "continuation": " invertebrates"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Answer:", "continuation": " insects"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Answer:", "continuation": " mammals"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-187", "query": "Question: small reptile's diet consists mostly of\nAnswer:", "choices": ["invertebrates", "insects", "mammals", "fish"], "gold": 1}, "request": {"context": "Answer:", "continuation": " fish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000228, "native_id": "9-187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Answer:", "continuation": " provide nice scents"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Answer:", "continuation": " be successfully fertilized"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Answer:", "continuation": " grow very tall"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1345", "query": "Question: The life work of a flower is to\nAnswer:", "choices": ["provide nice scents", "be successfully fertilized", "grow very tall", "look pretty"], "gold": 1}, "request": {"context": "Answer:", "continuation": " look pretty"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000229, "native_id": "1345", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the plastic water bottle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the backside of a person"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the hair on a doll"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-59", "query": "Question: Which of these would create the most sound if struck with a metal spoon?\nAnswer:", "choices": ["the plastic water bottle", "the backside of a person", "the hair on a doll", "the chassis of a car"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the chassis of a car"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000230, "native_id": "8-59", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Answer:", "continuation": " mud"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Answer:", "continuation": " snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "178", "query": "Question: During landslides there is often a lot of\nAnswer:", "choices": ["air", "mud", "snow", "wind"], "gold": 1}, "request": {"context": "Answer:", "continuation": " wind"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000231, "native_id": "178", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Answer:", "continuation": " A rusty fence"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Sleeping"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Drinking water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1186", "query": "Question: An example of a chemical reaction would be\nAnswer:", "choices": ["A rusty fence", "Sleeping", "Drinking water", "Rain"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000232, "native_id": "9-1186", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Answer:", "continuation": " equal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Answer:", "continuation": " inverse"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Answer:", "continuation": " direct"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "82", "query": "Question: The size of an object and the ability to see it more easily have what kind of relationship?\nAnswer:", "choices": ["equal", "inverse", "direct", "reverse"], "gold": 2}, "request": {"context": "Answer:", "continuation": " reverse"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000233, "native_id": "82", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Answer:", "continuation": " parasites, soybeans and flaxseeds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sea turtles, sharks and coral reefs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Answer:", "continuation": " spineless marine organisms, cartilaginous and gelatinous organisms"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-165", "query": "Question: Tuna primarily eat\nAnswer:", "choices": ["parasites, soybeans and flaxseeds", "sea turtles, sharks and coral reefs", "spineless marine organisms, cartilaginous and gelatinous organisms", "sea vegetables like kelp, Irish moss and Arame"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sea vegetables like kelp, Irish moss and Arame"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000234, "native_id": "8-165", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " waterfalls"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " blizzards"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " earthquakes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "404", "query": "Question: Evaporation of water can lead to\nAnswer:", "choices": ["waterfalls", "blizzards", "earthquakes", "hot springs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " hot springs"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000235, "native_id": "404", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Answer:", "continuation": " horseshoe"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Answer:", "continuation": " tire"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cotton shirt"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "279", "query": "Question: What is an electrical energy conductor?\nAnswer:", "choices": ["horseshoe", "tire", "cotton shirt", "maple branch"], "gold": 0}, "request": {"context": "Answer:", "continuation": " maple branch"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000236, "native_id": "279", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Answer:", "continuation": " warm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Answer:", "continuation": " frozen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Answer:", "continuation": " chilly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-532", "query": "Question: What kind of substance will cool when it touches a cold object?\nAnswer:", "choices": ["warm", "frozen", "chilly", "cold"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cold"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000237, "native_id": "9-532", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Answer:", "continuation": " may cause the object to grow in size"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Answer:", "continuation": " may cause the object to fall"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Answer:", "continuation": " may cause pieces to flake off the object"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "268", "query": "Question: Scraping an object\nAnswer:", "choices": ["may cause the object to grow in size", "may cause the object to fall", "may cause pieces to flake off the object", "may cause the object to snap in half"], "gold": 2}, "request": {"context": "Answer:", "continuation": " may cause the object to snap in half"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000238, "native_id": "268", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Answer:", "continuation": " moving a rope sideways"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Answer:", "continuation": " putting something in the air"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Answer:", "continuation": " moving things with wheels"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1018", "query": "Question: A pulley is used to lift a flag on a flagpole by\nAnswer:", "choices": ["moving a rope sideways", "putting something in the air", "moving things with wheels", "yanking string up a wheel"], "gold": 3}, "request": {"context": "Answer:", "continuation": " yanking string up a wheel"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000239, "native_id": "7-1018", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " emus"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " dogs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " squirrels"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1756", "query": "Question: Which animal lays eggs\nAnswer:", "choices": ["emus", "dogs", "squirrels", "giraffes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " giraffes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000240, "native_id": "1756", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " clothes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " food"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " forests"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1137", "query": "Question: Dunes can be made out of the same thing as\nAnswer:", "choices": ["clothes", "food", "forests", "castles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " castles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000241, "native_id": "1137", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Answer:", "continuation": " squinting in bright light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Answer:", "continuation": " inhaling and exhaling during sleep"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Answer:", "continuation": " blinking and gulping air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-203", "query": "Question: A learned behavior is exhibited when\nAnswer:", "choices": ["squinting in bright light", "inhaling and exhaling during sleep", "blinking and gulping air", "nailing up a picture frame"], "gold": 3}, "request": {"context": "Answer:", "continuation": " nailing up a picture frame"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000242, "native_id": "7-203", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Answer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Answer:", "continuation": " rubber"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "745", "query": "Question: Wax can be used similarly to\nAnswer:", "choices": ["wood", "rubber", "water", "metal"], "gold": 1}, "request": {"context": "Answer:", "continuation": " metal"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000243, "native_id": "745", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Answer:", "continuation": " get bigger yields"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Answer:", "continuation": " make things boring"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Answer:", "continuation": " keep things random"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-902", "query": "Question: Bill planted rapeseed in his field one year and soybeans the next in order to\nAnswer:", "choices": ["get bigger yields", "make things boring", "keep things random", "get smaller yields"], "gold": 0}, "request": {"context": "Answer:", "continuation": " get smaller yields"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000244, "native_id": "7-902", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " feathers slows heat transfer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the natural duck wax"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a synthetic thick liner"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1095", "query": "Question: What is the primary reason my duck feather filled jacket works well against the snow\nAnswer:", "choices": ["feathers slows heat transfer", "the natural duck wax", "a synthetic thick liner", "small flexible solar panels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " small flexible solar panels"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000245, "native_id": "1095", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a charge for appliances"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Answer:", "continuation": " large quantities of soup"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Answer:", "continuation": " large schools of fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-163", "query": "Question: Turbines churning seawater can be used to produce what?\nAnswer:", "choices": ["a charge for appliances", "large quantities of soup", "large schools of fish", "creating some sharp cheese"], "gold": 0}, "request": {"context": "Answer:", "continuation": " creating some sharp cheese"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000246, "native_id": "7-163", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Answer:", "continuation": " weight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Answer:", "continuation": " height"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Answer:", "continuation": " hotness"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-858", "query": "Question: What will increase when a substance absorbs solar energy?\nAnswer:", "choices": ["weight", "height", "hotness", "nutrition"], "gold": 2}, "request": {"context": "Answer:", "continuation": " nutrition"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000247, "native_id": "9-858", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Answer:", "continuation": " planets"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Answer:", "continuation": " thoughts"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Answer:", "continuation": " automobile"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1530", "query": "Question: Which of these travels through the air?\nAnswer:", "choices": ["planets", "thoughts", "automobile", "music"], "gold": 3}, "request": {"context": "Answer:", "continuation": " music"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000248, "native_id": "1530", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " chips"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " grass"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-993", "query": "Question: A company makes notebooks for college courses, so their main material is\nAnswer:", "choices": ["chips", "water", "grass", "trees"], "gold": 3}, "request": {"context": "Answer:", "continuation": " trees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000249, "native_id": "9-993", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Answer:", "continuation": " cirrus clouds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Answer:", "continuation": " cumulus clouds"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Answer:", "continuation": " hail stones"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-340", "query": "Question: Rain is usually guaranteed when all are present but\nAnswer:", "choices": ["cirrus clouds", "cumulus clouds", "hail stones", "direct sunshine"], "gold": 3}, "request": {"context": "Answer:", "continuation": " direct sunshine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000250, "native_id": "8-340", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Walking"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Landfills"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "3", "query": "Question: Sources of air pollution are\nAnswer:", "choices": ["Walking", "Landfills", "Water", "Chips"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Chips"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000251, "native_id": "3", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " plastic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " metal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1074", "query": "Question: What are iron nails made out of?\nAnswer:", "choices": ["wood", "plastic", "metal", "glass"], "gold": 2}, "request": {"context": "Answer:", "continuation": " glass"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000252, "native_id": "1074", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Answer:", "continuation": " combining"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Answer:", "continuation": " drinking"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Subtracting"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-431", "query": "Question: Putting one kind of soda into the same cup as another kind of soda is doing what to the substances?\nAnswer:", "choices": ["combining", "drinking", "Subtracting", "throwing"], "gold": 0}, "request": {"context": "Answer:", "continuation": " throwing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000253, "native_id": "9-431", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " life cycle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " transformation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " recycling process"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-638", "query": "Question: A moth leaving it's cocoon is the final step in a\nAnswer:", "choices": ["life cycle", "transformation", "recycling process", "chemical reaction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " chemical reaction"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000254, "native_id": "9-638", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Answer:", "continuation": " ice and snow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Answer:", "continuation": " sticks and stones"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Answer:", "continuation": " snow and water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-352", "query": "Question: Which of these combinations would be desired if someone wanted to make a cutting implement that lasts a long time?\nAnswer:", "choices": ["ice and snow", "sticks and stones", "snow and water", "iron and carbon"], "gold": 3}, "request": {"context": "Answer:", "continuation": " iron and carbon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000255, "native_id": "9-352", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a bicycle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a motorcycle"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a propane grill"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "226", "query": "Question: Which of the following is powered the same way an electric car is?\nAnswer:", "choices": ["a bicycle", "a motorcycle", "a propane grill", "a blender"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a blender"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000256, "native_id": "226", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Answer:", "continuation": " reprocessable"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Answer:", "continuation": " plastic"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Answer:", "continuation": " liquid"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-132", "query": "Question: Aluminum is what?\nAnswer:", "choices": ["reprocessable", "plastic", "liquid", "absorbent"], "gold": 0}, "request": {"context": "Answer:", "continuation": " absorbent"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000257, "native_id": "9-132", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Answer:", "continuation": " wear a Christmas sweater"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Drink a hot chocolate"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Go for a run"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-222", "query": "Question: To get warm frogs can\nAnswer:", "choices": ["wear a Christmas sweater", "Drink a hot chocolate", "Go for a run", "sit under a lamp"], "gold": 3}, "request": {"context": "Answer:", "continuation": " sit under a lamp"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000258, "native_id": "9-222", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it exhibits absolute rigidity"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it is a solid mass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-105", "query": "Question: if you put wine from a jug into a thin bottle, how come it conforms?\nAnswer:", "choices": ["it exhibits absolute rigidity", "it is a solid mass", "all of these", "it is a variable substance"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it is a variable substance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000259, "native_id": "9-105", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Answer:", "continuation": " ice"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Answer:", "continuation": " dirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-459", "query": "Question: Earthworms create tunnels in\nAnswer:", "choices": ["ice", "dirt", "water", "concrete"], "gold": 1}, "request": {"context": "Answer:", "continuation": " concrete"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000260, "native_id": "7-459", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Answer:", "continuation": " regurgitation"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Answer:", "continuation": " decomposition"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Answer:", "continuation": " recycling"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-881", "query": "Question: when worms return nutrients from dead organisms to the soil by eating them it is known as\nAnswer:", "choices": ["regurgitation", "decomposition", "recycling", "burial"], "gold": 1}, "request": {"context": "Answer:", "continuation": " burial"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000261, "native_id": "9-881", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Answer:", "continuation": " gold jewelry"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Answer:", "continuation": " wooden fence"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Answer:", "continuation": " brass doorknob"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "280", "query": "Question: The aluminum cans were much hotter than the\nAnswer:", "choices": ["gold jewelry", "wooden fence", "brass doorknob", "steel pole"], "gold": 1}, "request": {"context": "Answer:", "continuation": " steel pole"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000262, "native_id": "280", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sunlight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " oxygen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "187", "query": "Question: Which of the following is not an input in photosynthesis?\nAnswer:", "choices": ["sunlight", "oxygen", "water", "carbon dioxide"], "gold": 1}, "request": {"context": "Answer:", "continuation": " carbon dioxide"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000263, "native_id": "187", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a hammer head shark"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a big tilapia fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a prickly horned male lizard"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-253", "query": "Question: which of these are you most likely to find in a desert?\nAnswer:", "choices": ["a hammer head shark", "a big tilapia fish", "a prickly horned male lizard", "none of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000264, "native_id": "8-253", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it will be fine"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it will be happy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it will be comfortable"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-482", "query": "Question: If your dog sits in an oxygen deficient chamber, what happens?\nAnswer:", "choices": ["it will be fine", "it will be happy", "it will be comfortable", "It will pass out"], "gold": 3}, "request": {"context": "Answer:", "continuation": " It will pass out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000265, "native_id": "9-482", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Answer:", "continuation": " solar-rechargeable battery"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a wedge"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a magnet"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "496", "query": "Question: Which of the following can be used to turn on an electrical device?\nAnswer:", "choices": ["solar-rechargeable battery", "a wedge", "a magnet", "pressure gauge"], "gold": 0}, "request": {"context": "Answer:", "continuation": " pressure gauge"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000266, "native_id": "496", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The Amazon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The Nile"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The Indian"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "630", "query": "Question: Which of the following contains large amounts of salt water?\nAnswer:", "choices": ["The Amazon", "The Nile", "The Indian", "The Mississippi"], "gold": 2}, "request": {"context": "Answer:", "continuation": " The Mississippi"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000267, "native_id": "630", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the butterfly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the sloth"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the praying mantis"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-16", "query": "Question: The nimbleness of this animal is a key adaption that allows it to escape attacks from predators:\nAnswer:", "choices": ["the butterfly", "the sloth", "the praying mantis", "the antelope"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the antelope"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000268, "native_id": "9-16", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Answer:", "continuation": " media is mainly in English"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Answer:", "continuation": " school is in English"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Answer:", "continuation": " she was genetically predisposed"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-986", "query": "Question: A person speaks English as her first language because\nAnswer:", "choices": ["media is mainly in English", "school is in English", "she was genetically predisposed", "she watched her parents speak"], "gold": 3}, "request": {"context": "Answer:", "continuation": " she watched her parents speak"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000269, "native_id": "7-986", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they consume plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they eat cows"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " plant reproduction parts"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-787", "query": "Question: what are eaten by honey producing insects?\nAnswer:", "choices": ["they consume plants", "they eat cows", "plant reproduction parts", "they eat flowers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they eat flowers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000270, "native_id": "7-787", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Answer:", "continuation": " our celestial fireball"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Answer:", "continuation": " solar flares"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Answer:", "continuation": " gamma rays"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-181", "query": "Question: The Earth's closest heat source is\nAnswer:", "choices": ["our celestial fireball", "solar flares", "gamma rays", "big bang"], "gold": 0}, "request": {"context": "Answer:", "continuation": " big bang"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000271, "native_id": "9-181", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sand is always moving"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sandstorms create ripples in sand"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sand is easy to move through"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1240", "query": "Question: How can we see how wind effects sand?\nAnswer:", "choices": ["sand is always moving", "sandstorms create ripples in sand", "sand is easy to move through", "beaches often have waves in the sand"], "gold": 1}, "request": {"context": "Answer:", "continuation": " beaches often have waves in the sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000272, "native_id": "1240", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an organism that releases carbon dioxide"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an organism that absorbs water through it's branches"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an organism that absorbs oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "474", "query": "Question: Fruit comes from what source\nAnswer:", "choices": ["an organism that releases carbon dioxide", "an organism that absorbs water through it's branches", "an organism that absorbs oxygen", "an organism that absorbs water through it's roots"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an organism that absorbs water through it's roots"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000273, "native_id": "474", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Answer:", "continuation": " beside a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in the grass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Answer:", "continuation": " on a stump"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1274", "query": "Question: In which location would a groundhog hide from a wolf?\nAnswer:", "choices": ["beside a tree", "in the grass", "on a stump", "under the ground"], "gold": 3}, "request": {"context": "Answer:", "continuation": " under the ground"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000274, "native_id": "1274", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " leads to more habitats for animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " decreases the chance of erosion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " increases the number of trees in the forest"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1531", "query": "Question: Cutting down trees in a forest\nAnswer:", "choices": ["leads to more habitats for animals", "decreases the chance of erosion", "increases the number of trees in the forest", "leads to less habitats for animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " leads to less habitats for animals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000275, "native_id": "1531", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " animals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " resistance"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-321", "query": "Question: I chipped away at a toy doll and the surface became really rough, when I rub it against a piece of wood that will create an increase in\nAnswer:", "choices": ["animals", "resistance", "water", "sunshine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sunshine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000276, "native_id": "8-321", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Answer:", "continuation": " because it's overpopulated with polar bears"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Answer:", "continuation": " because it's covered in white lilies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Answer:", "continuation": " because it's blanketed in crystalline ice water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1321", "query": "Question: The arctic is white in coloring\nAnswer:", "choices": ["because it's overpopulated with polar bears", "because it's covered in white lilies", "because it's blanketed in crystalline ice water", "because it's gets so little sunlight"], "gold": 2}, "request": {"context": "Answer:", "continuation": " because it's gets so little sunlight"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000277, "native_id": "1321", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Lots of meat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Lots of love"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Vitamin supplements"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-51", "query": "Question: What would help to ensure that your dog remains free from hypothermia in January in Alaska?\nAnswer:", "choices": ["Lots of meat", "Lots of love", "Vitamin supplements", "An insulated room"], "gold": 3}, "request": {"context": "Answer:", "continuation": " An insulated room"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000278, "native_id": "9-51", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Answer:", "continuation": " fleas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Answer:", "continuation": " crawlies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Answer:", "continuation": " gummy worms"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-685", "query": "Question: The majority of a lizard's diet consists of\nAnswer:", "choices": ["fleas", "crawlies", "gummy worms", "berries"], "gold": 1}, "request": {"context": "Answer:", "continuation": " berries"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000279, "native_id": "7-685", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " nutrient making process"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the breathing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the respiration"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-59", "query": "Question: What food production happens in a leaf?\nAnswer:", "choices": ["nutrient making process", "the breathing", "the respiration", "the digestion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the digestion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000280, "native_id": "7-59", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a nice cool breeze"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fresh soil with manure"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a regular source of saltwater"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-270", "query": "Question: Plants are unable to grow if they have zero access to\nAnswer:", "choices": ["a nice cool breeze", "fresh soil with manure", "a regular source of saltwater", "needs required for creating chlorophyll"], "gold": 3}, "request": {"context": "Answer:", "continuation": " needs required for creating chlorophyll"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000281, "native_id": "7-270", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " length of hair"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " number of friends"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " number of nails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-736", "query": "Question: Which characteristic did a person inherit?\nAnswer:", "choices": ["length of hair", "number of friends", "number of nails", "length of shirt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " length of shirt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000282, "native_id": "7-736", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Answer:", "continuation": " rain clouds and ozone layer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Answer:", "continuation": " lakes, ponds and shellfish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Answer:", "continuation": " greenhouse gases and algae"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-186", "query": "Question: Selective deforestation has a negative impact on\nAnswer:", "choices": ["rain clouds and ozone layer", "lakes, ponds and shellfish", "greenhouse gases and algae", "living organisms in ecosystem"], "gold": 3}, "request": {"context": "Answer:", "continuation": " living organisms in ecosystem"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000283, "native_id": "8-186", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " in a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " under a mountain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " in the air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "224", "query": "Question: Where would you find a mine?\nAnswer:", "choices": ["in a tree", "under a mountain", "in the air", "in the water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " in the water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000284, "native_id": "224", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Seeing a solar eclipse"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Using their turn signals"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Driving the speed limit"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-206", "query": "Question: What can cause people to crash their car?\nAnswer:", "choices": ["Seeing a solar eclipse", "Using their turn signals", "Driving the speed limit", "Keeping their eyes on the road"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Keeping their eyes on the road"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000285, "native_id": "8-206", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Answer:", "continuation": " frigid waters"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Answer:", "continuation": " tepid seas"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Answer:", "continuation": " glacial environments"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-190", "query": "Question: Coral grows in\nAnswer:", "choices": ["frigid waters", "tepid seas", "glacial environments", "jungle forests"], "gold": 1}, "request": {"context": "Answer:", "continuation": " jungle forests"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000286, "native_id": "8-190", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Answer:", "continuation": " certain things may be featured"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Answer:", "continuation": " certain features may be predicted"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Answer:", "continuation": " certain traits may be given"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-334", "query": "Question: A Punnett square is used to identify the percent chance of a trait being passed down from a parent to its offspring, so\nAnswer:", "choices": ["certain things may be featured", "certain features may be predicted", "certain traits may be given", "certain features may be guaranteed"], "gold": 1}, "request": {"context": "Answer:", "continuation": " certain features may be guaranteed"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000287, "native_id": "7-334", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " longer antlers"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " food for itself"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " baby deer"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-853", "query": "Question: A deer is eating in a field, and wants more food. Regardless of how hard the deer tries, the deer is unable to produce\nAnswer:", "choices": ["longer antlers", "food for itself", "baby deer", "urine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " urine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000288, "native_id": "9-853", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Answer:", "continuation": " community concerns over environmental impact"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Answer:", "continuation": " better air and soil quality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Answer:", "continuation": " higher value on land parcels"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-367", "query": "Question: Building new areas to dispose of refuse may lead to\nAnswer:", "choices": ["community concerns over environmental impact", "better air and soil quality", "higher value on land parcels", "improvement in water supply"], "gold": 0}, "request": {"context": "Answer:", "continuation": " improvement in water supply"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000289, "native_id": "8-367", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Answer:", "continuation": " causes puddles to become dried out mud"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Answer:", "continuation": " causes fields of crops to grow faster"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Answer:", "continuation": " causes flowers to bloom abundantly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1047", "query": "Question: Evaporation\nAnswer:", "choices": ["causes puddles to become dried out mud", "causes fields of crops to grow faster", "causes flowers to bloom abundantly", "fills up irrigation ponds"], "gold": 0}, "request": {"context": "Answer:", "continuation": " fills up irrigation ponds"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000290, "native_id": "1047", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " pythons"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " salmon"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " robins"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-454", "query": "Question: A field begins to bloom and blossom and plants need to be pollinated. In order to spread seeds, plants will most rely on\nAnswer:", "choices": ["pythons", "salmon", "robins", "craters"], "gold": 2}, "request": {"context": "Answer:", "continuation": " craters"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000291, "native_id": "9-454", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Answer:", "continuation": " airplane"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Answer:", "continuation": " car"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Answer:", "continuation": " mammal"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1572", "query": "Question: Which item urinates?\nAnswer:", "choices": ["airplane", "car", "mammal", "boat"], "gold": 2}, "request": {"context": "Answer:", "continuation": " boat"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000292, "native_id": "1572", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Warm breath fogging up a mirror"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Morning dew drying on the grass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Answer:", "continuation": " The water level in a glass decreasing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-373", "query": "Question: All of the following are examples of evaporation apart from\nAnswer:", "choices": ["Warm breath fogging up a mirror", "Morning dew drying on the grass", "The water level in a glass decreasing", "Sweat drying on skin"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Sweat drying on skin"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000293, "native_id": "8-373", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Answer:", "continuation": " colors"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the sun"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-772", "query": "Question: The lunar cycle also changes\nAnswer:", "choices": ["water", "colors", "the sun", "planets"], "gold": 0}, "request": {"context": "Answer:", "continuation": " planets"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000294, "native_id": "9-772", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " is nutritionally self sustaining"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " will die faster than other organisms"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " will need help sustaining strength"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1852", "query": "Question: An organism that makes food for itself\nAnswer:", "choices": ["is nutritionally self sustaining", "will die faster than other organisms", "will need help sustaining strength", "is reliant on other organisms for assistance"], "gold": 0}, "request": {"context": "Answer:", "continuation": " is reliant on other organisms for assistance"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000295, "native_id": "1852", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " food"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " nutrients"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1090", "query": "Question: What does the respiratory system transfer to the circulatory system?\nAnswer:", "choices": ["food", "water", "nutrients", "O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " O"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000296, "native_id": "9-1090", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " burn out"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " charge itself"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " loop endlessly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-769", "query": "Question: In a closed circuit, electricity will\nAnswer:", "choices": ["burn out", "charge itself", "loop endlessly", "resist flow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " resist flow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000297, "native_id": "7-769", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Answer:", "continuation": " mother"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Answer:", "continuation": " grandfather"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Answer:", "continuation": " daughter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-478", "query": "Question: A Punnett square can be used to calculate the chance of a trait being passed to someone's\nAnswer:", "choices": ["mother", "grandfather", "daughter", "father"], "gold": 2}, "request": {"context": "Answer:", "continuation": " father"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000298, "native_id": "9-478", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Answer:", "continuation": " richer"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Answer:", "continuation": " dryer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sandy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "448", "query": "Question: They looked where the log decayed to garden as it would leave the earth\nAnswer:", "choices": ["richer", "dryer", "sandy", "harder"], "gold": 0}, "request": {"context": "Answer:", "continuation": " harder"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000299, "native_id": "448", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " in freezing cold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a high temperature"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " in any temperature"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-417", "query": "Question: what kind of temperature causes fur shedding?\nAnswer:", "choices": ["in freezing cold", "a high temperature", "in any temperature", "a low temperature"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a low temperature"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000300, "native_id": "7-417", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Answer:", "continuation": " wet items"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Answer:", "continuation": " cold items"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-108", "query": "Question: exposure to fire could result in\nAnswer:", "choices": ["wet items", "cold items", "none of these", "combusted items"], "gold": 3}, "request": {"context": "Answer:", "continuation": " combusted items"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000301, "native_id": "7-108", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Answer:", "continuation": " more stars are visible on clear nights"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Answer:", "continuation": " cloud cover protects from sunburn"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Answer:", "continuation": " clear days will be warmer"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1506", "query": "Question: What is an example of clear weather meaning sunny weather?\nAnswer:", "choices": ["more stars are visible on clear nights", "cloud cover protects from sunburn", "clear days will be warmer", "fewer clouds allow for more sunlight"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fewer clouds allow for more sunlight"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000302, "native_id": "1506", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a wick"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a funnel"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a knife"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1712", "query": "Question: The special tissues in plants that transport minerals throughout the plant are similar to\nAnswer:", "choices": ["a wick", "a funnel", "a knife", "a whisk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a whisk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000303, "native_id": "1712", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Answer:", "continuation": " to western Canada shoreline"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Answer:", "continuation": " to the lower pole"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Answer:", "continuation": " to the upper pole"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-312", "query": "Question: a compass is a kind of tool for determining direction by pointing\nAnswer:", "choices": ["to western Canada shoreline", "to the lower pole", "to the upper pole", "directly to the equator"], "gold": 2}, "request": {"context": "Answer:", "continuation": " directly to the equator"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000304, "native_id": "8-312", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Answer:", "continuation": " iron saw"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Answer:", "continuation": " aluminium saw"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Answer:", "continuation": " plastic saw"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-776", "query": "Question: Which of these saws will last longer?\nAnswer:", "choices": ["iron saw", "aluminium saw", "plastic saw", "wooden saw"], "gold": 0}, "request": {"context": "Answer:", "continuation": " wooden saw"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000305, "native_id": "9-776", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the single moon close to us"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the orion star cluster"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the sun that shines all day"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-279", "query": "Question: although there are many stars visible in the night sky, which is most visible in the day?\nAnswer:", "choices": ["the single moon close to us", "the orion star cluster", "the sun that shines all day", "all of these"], "gold": 2}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000306, "native_id": "8-279", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Answer:", "continuation": " frozen streams of water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Answer:", "continuation": " large bowl shaped cavities"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Answer:", "continuation": " caves formed by solar winds"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-621", "query": "Question: The moon is known for having what feature?\nAnswer:", "choices": ["frozen streams of water", "large bowl shaped cavities", "caves formed by solar winds", "groups of large trees"], "gold": 1}, "request": {"context": "Answer:", "continuation": " groups of large trees"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000307, "native_id": "9-621", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a bug's treasure"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a cow's treasure"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a plant's treasure"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1823", "query": "Question: Cellular respiration's trash is\nAnswer:", "choices": ["a bug's treasure", "a cow's treasure", "a plant's treasure", "everyone's trash"], "gold": 2}, "request": {"context": "Answer:", "continuation": " everyone's trash"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000308, "native_id": "1823", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " swimming in a lake"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " building a new subdivision"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " dancing in a field"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-735", "query": "Question: Which of the following human activities can lead to a change in the local ecosystem?\nAnswer:", "choices": ["swimming in a lake", "building a new subdivision", "dancing in a field", "going for a hike"], "gold": 1}, "request": {"context": "Answer:", "continuation": " going for a hike"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000309, "native_id": "9-735", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " using heat to produce motion"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " using calories to produce motion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " using wings to produce heat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1170", "query": "Question: A bird that takes off flying is\nAnswer:", "choices": ["using heat to produce motion", "using calories to produce motion", "using wings to produce heat", "using calories to produce energy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " using calories to produce energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000310, "native_id": "7-1170", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " H2O"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " CO2"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " NaCl"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1500", "query": "Question: The leading cause of soil and rock erosion is\nAnswer:", "choices": ["H2O", "CO2", "NaCl", "Fe"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Fe"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000311, "native_id": "1500", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Organic corn"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Conventional corn"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Organic potato"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "342", "query": "Question: Which of these foods might have a negative impact on humans?\nAnswer:", "choices": ["Organic corn", "Conventional corn", "Organic potato", "Organic Apples"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Organic Apples"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000312, "native_id": "342", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Answer:", "continuation": " to test heat"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Answer:", "continuation": " for wind speed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it measures distance"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-356", "query": "Question: What kind of implement is a compass?\nAnswer:", "choices": ["to test heat", "for wind speed", "it measures distance", "it shows direction"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it shows direction"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000313, "native_id": "7-356", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Answer:", "continuation": " flowers"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a hive"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a stream"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "78", "query": "Question: Nectar is taken to\nAnswer:", "choices": ["flowers", "a hive", "a stream", "a nest"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a nest"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000314, "native_id": "78", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the cloud in the sky"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the nearest star sulfur burning heavenly body"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-520", "query": "Question: what is the closest source of plasma to our planet?\nAnswer:", "choices": ["all of these", "the cloud in the sky", "the nearest star sulfur burning heavenly body", "the bare moon surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the bare moon surface"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000315, "native_id": "9-520", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Answer:", "continuation": " closing the sails quickly"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Answer:", "continuation": " setting out to sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Answer:", "continuation": " making an adjustment to the rudder"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-653", "query": "Question: A sailor needs to navigate to the shore, and does this by\nAnswer:", "choices": ["closing the sails quickly", "setting out to sea", "making an adjustment to the rudder", "taking the afternoon off"], "gold": 2}, "request": {"context": "Answer:", "continuation": " taking the afternoon off"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000316, "native_id": "7-653", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Answer:", "continuation": " chicken"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Answer:", "continuation": " pig"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Answer:", "continuation": " butterfly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1112", "query": "Question: A bat starts its life similarly to a\nAnswer:", "choices": ["chicken", "pig", "butterfly", "duck"], "gold": 1}, "request": {"context": "Answer:", "continuation": " duck"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000317, "native_id": "1112", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Answer:", "continuation": " big"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Answer:", "continuation": " quick"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Answer:", "continuation": " slow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-152", "query": "Question: In order to catch a rabbit, a predator must be\nAnswer:", "choices": ["big", "quick", "slow", "small"], "gold": 1}, "request": {"context": "Answer:", "continuation": " small"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000318, "native_id": "9-152", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Answer:", "continuation": " prey"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Answer:", "continuation": " predator"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Answer:", "continuation": " herbivore"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-552", "query": "Question: If a bird is a carnivore, then it is likely a(n)\nAnswer:", "choices": ["prey", "predator", "herbivore", "canary"], "gold": 1}, "request": {"context": "Answer:", "continuation": " canary"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000319, "native_id": "9-552", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the Sahara"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the mountains"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the ocean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-262", "query": "Question: A warm-weather organism can be found in\nAnswer:", "choices": ["the Sahara", "the mountains", "the ocean", "the sewers"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the sewers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000320, "native_id": "7-262", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it stays large"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it grows larger"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it gets bigger"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-683", "query": "Question: When approaching an elephant from a great distance,\nAnswer:", "choices": ["it stays large", "it grows larger", "it gets bigger", "it looks bigger"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it looks bigger"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000321, "native_id": "7-683", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Answer:", "continuation": " light waves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Answer:", "continuation": " eating wheat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Answer:", "continuation": " photosynthesis"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "276", "query": "Question: What would cause a human to grow?\nAnswer:", "choices": ["light waves", "eating wheat", "photosynthesis", "marching"], "gold": 1}, "request": {"context": "Answer:", "continuation": " marching"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000322, "native_id": "276", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " lots of snow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " many people"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " less water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-855", "query": "Question: A saguaro has adaptations for an environment with\nAnswer:", "choices": ["lots of snow", "many people", "less water", "more water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " more water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000323, "native_id": "7-855", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a feeder at this house"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the birds no longer like feeders"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the size of the feeder"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "664", "query": "Question: There are less hummingbirds by this house than before because of\nAnswer:", "choices": ["a feeder at this house", "the birds no longer like feeders", "the size of the feeder", "a feeder at another house"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a feeder at another house"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000324, "native_id": "664", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " water lilies"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " guppies"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sea life"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-883", "query": "Question: the  oceans are full of\nAnswer:", "choices": ["water lilies", "guppies", "sea life", "fresh water"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fresh water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000325, "native_id": "9-883", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sitting on the table"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Answer:", "continuation": " attached to the wall"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Answer:", "continuation": " attached to an extension cord"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-550", "query": "Question: A light was off because the cord was\nAnswer:", "choices": ["sitting on the table", "attached to the wall", "attached to an extension cord", "attached to a battery pack"], "gold": 0}, "request": {"context": "Answer:", "continuation": " attached to a battery pack"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000326, "native_id": "9-550", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in sticks"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in pebbles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in sand"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-493", "query": "Question: In the hottest months in the hottest desert, creatures such as birds may find water to drink\nAnswer:", "choices": ["in sticks", "in pebbles", "in sand", "in spiked plants"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in spiked plants"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000327, "native_id": "8-493", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " look fancy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " random"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " blend in"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-257", "query": "Question: Why might a polar bear grow white hair?\nAnswer:", "choices": ["look fancy", "random", "blend in", "stand out"], "gold": 2}, "request": {"context": "Answer:", "continuation": " stand out"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000328, "native_id": "9-257", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Answer:", "continuation": " discourages pests from landing on leaves"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Answer:", "continuation": " allows plants to move carbon dioxide from root to stems"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Answer:", "continuation": " carries seedlings from roots to leaves"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1239", "query": "Question: Xylem\nAnswer:", "choices": ["discourages pests from landing on leaves", "allows plants to move carbon dioxide from root to stems", "carries seedlings from roots to leaves", "allows plants to move rain thru their systems"], "gold": 3}, "request": {"context": "Answer:", "continuation": " allows plants to move rain thru their systems"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000329, "native_id": "1239", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Answer:", "continuation": " ramen"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Answer:", "continuation": " salad"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Answer:", "continuation": " ice cream"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "869", "query": "Question: A food that is a source of heat is\nAnswer:", "choices": ["ramen", "salad", "ice cream", "sushi"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sushi"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000330, "native_id": "869", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " contaminates may be destroyed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " bacterial can grow more rapidly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " viruses may be picked up"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1105", "query": "Question: When heat is added to something\nAnswer:", "choices": ["contaminates may be destroyed", "bacterial can grow more rapidly", "viruses may be picked up", "the thing loses energy"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the thing loses energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000331, "native_id": "7-1105", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a part of the nervous system"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an organ that filters toxins"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Answer:", "continuation": " a self-healing protector from germs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "597", "query": "Question: The heart is an example of\nAnswer:", "choices": ["a part of the nervous system", "an organ that filters toxins", "a self-healing protector from germs", "something protected by the skeletal system"], "gold": 3}, "request": {"context": "Answer:", "continuation": " something protected by the skeletal system"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000332, "native_id": "597", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an animal herded by sheep dogs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the animal with a starring role in Bambi"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Answer:", "continuation": " animals known for their memory"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "385", "query": "Question: Prey are eaten by\nAnswer:", "choices": ["an animal herded by sheep dogs", "the animal with a starring role in Bambi", "animals known for their memory", "the fastest mammal with four legs"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the fastest mammal with four legs"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000333, "native_id": "385", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a snake"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a cardinal"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a spider"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1301", "query": "Question: A dog is warm-blooded just like\nAnswer:", "choices": ["a snake", "a cardinal", "a spider", "a scorpion"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a scorpion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000334, "native_id": "1301", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Answer:", "continuation": " radiation"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Answer:", "continuation": " acoustic energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Answer:", "continuation": " vibrations"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-893", "query": "Question: A flashlight will need this in order to radiate photons:\nAnswer:", "choices": ["radiation", "acoustic energy", "vibrations", "electron flow"], "gold": 3}, "request": {"context": "Answer:", "continuation": " electron flow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000335, "native_id": "9-893", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Answer:", "continuation": " where you're going"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Answer:", "continuation": " distance traveled"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Answer:", "continuation": " distance to travel"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-369", "query": "Question: to find out how fast you are going you first need to know\nAnswer:", "choices": ["where you're going", "distance traveled", "distance to travel", "home location"], "gold": 1}, "request": {"context": "Answer:", "continuation": " home location"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000336, "native_id": "9-369", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Answer:", "continuation": " weaken"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Answer:", "continuation": " strengthen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Answer:", "continuation": " coexist"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1026", "query": "Question: A small lamb, two days old, is walking with its mother. The mother feels ill, so refuses food, which dries up her milk production. The lack of lactation causes the lamb to\nAnswer:", "choices": ["weaken", "strengthen", "coexist", "thrive"], "gold": 0}, "request": {"context": "Answer:", "continuation": " thrive"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000337, "native_id": "9-1026", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a dinner jacket"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a diving suit"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a warm coat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-424", "query": "Question: When trying to find fresh clams for dinner, a hungry person would don\nAnswer:", "choices": ["a dinner jacket", "a diving suit", "a warm coat", "a dress suit"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a dress suit"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000338, "native_id": "7-424", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Answer:", "continuation": " B and D"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Double Fur Coats"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Cold blooded"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-259", "query": "Question: How do polar bears survive the cold?\nAnswer:", "choices": ["B and D", "Double Fur Coats", "Cold blooded", "Compact ears"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Compact ears"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000339, "native_id": "9-259", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " floods"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " wind"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " chill"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-783", "query": "Question: A solid is likely to form in extreme\nAnswer:", "choices": ["floods", "wind", "chill", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000340, "native_id": "9-783", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " catching prey"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " aquatic speed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " flying"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1088", "query": "Question: What are the feet of Dendrocygna autumnalis designed for?\nAnswer:", "choices": ["catching prey", "aquatic speed", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " walking"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000341, "native_id": "1088", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an oven is preheated and the pilot light is lit"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a match is lit to light a cigarette"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a lit candle in a window signalling to someone"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1387", "query": "Question: What is an example of fire giving off light?\nAnswer:", "choices": ["an oven is preheated and the pilot light is lit", "a match is lit to light a cigarette", "a lit candle in a window signalling to someone", "a fire that was put out to send smoke signals"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a fire that was put out to send smoke signals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000342, "native_id": "1387", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Answer:", "continuation": " directing oxygen from lungs to other organs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Answer:", "continuation": " pushing air through lungs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Answer:", "continuation": " moving air in a room"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1062", "query": "Question: The respiratory system works by\nAnswer:", "choices": ["directing oxygen from lungs to other organs", "pushing air through lungs", "moving air in a room", "making air quality better"], "gold": 0}, "request": {"context": "Answer:", "continuation": " making air quality better"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000343, "native_id": "7-1062", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in the ocean"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in human homes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in landfills"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "676", "query": "Question: Animals have more fat\nAnswer:", "choices": ["in the ocean", "in human homes", "in landfills", "in polar areas"], "gold": 3}, "request": {"context": "Answer:", "continuation": " in polar areas"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000344, "native_id": "676", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Answer:", "continuation": " poison"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Answer:", "continuation": " its appearance"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Answer:", "continuation": " speed"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1998", "query": "Question: A stick bug uses what to protect itself from predators?\nAnswer:", "choices": ["poison", "its appearance", "speed", "hearing"], "gold": 1}, "request": {"context": "Answer:", "continuation": " hearing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000345, "native_id": "1998", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a simple alcohol"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " glass"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1698", "query": "Question: Corn is sometimes used to make\nAnswer:", "choices": ["a simple alcohol", "water", "glass", "milk"], "gold": 0}, "request": {"context": "Answer:", "continuation": " milk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000346, "native_id": "1698", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " heat energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " light energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " color energy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "490", "query": "Question: The inside of the Thanksgiving turkey is white instead of pink because of\nAnswer:", "choices": ["heat energy", "light energy", "color energy", "color transfusion"], "gold": 0}, "request": {"context": "Answer:", "continuation": " color transfusion"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000347, "native_id": "490", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Answer:", "continuation": " reproduction ?"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Answer:", "continuation": " pet store sale"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a begging child"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "844", "query": "Question: Little puppies are a result of:\nAnswer:", "choices": ["reproduction ?", "pet store sale", "a begging child", "evolution"], "gold": 0}, "request": {"context": "Answer:", "continuation": " evolution"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000348, "native_id": "844", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " hamburger"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " moles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " alfalfa"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1795", "query": "Question: What would a Jersey most likely be fed?\nAnswer:", "choices": ["hamburger", "moles", "alfalfa", "cow"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000349, "native_id": "1795", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Answer:", "continuation": " coal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Answer:", "continuation": " windmill"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Answer:", "continuation": " lithium batteries"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1508", "query": "Question: Which of these energy sources generates the least amount of pollution?\nAnswer:", "choices": ["coal", "windmill", "lithium batteries", "gasoline"], "gold": 1}, "request": {"context": "Answer:", "continuation": " gasoline"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000350, "native_id": "1508", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Answer:", "continuation": " trees"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Answer:", "continuation": " mountains"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Answer:", "continuation": " lakes"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-289", "query": "Question: Grass snakes live in what?\nAnswer:", "choices": ["trees", "mountains", "lakes", "turf"], "gold": 3}, "request": {"context": "Answer:", "continuation": " turf"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000351, "native_id": "9-289", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Answer:", "continuation": " mate"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Answer:", "continuation": " feed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Answer:", "continuation": " hide"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-668", "query": "Question: Cephalopod ink is by octopuses to\nAnswer:", "choices": ["mate", "feed", "hide", "play"], "gold": 2}, "request": {"context": "Answer:", "continuation": " play"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000352, "native_id": "9-668", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " reverberate when touched together"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " vibrate when next to each other"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " shake around when near"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-364", "query": "Question: Tapping a drumstick to a drum will\nAnswer:", "choices": ["reverberate when touched together", "vibrate when next to each other", "shake around when near", "put each other down"], "gold": 0}, "request": {"context": "Answer:", "continuation": " put each other down"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000353, "native_id": "7-364", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Answer:", "continuation": " June"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Answer:", "continuation": " March"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Answer:", "continuation": " September"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1271", "query": "Question: Snow is more likely to fall two months before\nAnswer:", "choices": ["June", "March", "September", "December"], "gold": 1}, "request": {"context": "Answer:", "continuation": " December"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000354, "native_id": "1271", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Scorpion"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " House Fly"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Cricket"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1117", "query": "Question: If I want to avoid being dinner for some type of frog what should I reincarnate as?\nAnswer:", "choices": ["Scorpion", "House Fly", "Cricket", "Moth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Moth"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000355, "native_id": "9-1117", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Answer:", "continuation": " January"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Answer:", "continuation": " July"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Answer:", "continuation": " May"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "35", "query": "Question: Dead plants are easier to find in\nAnswer:", "choices": ["January", "July", "May", "September"], "gold": 0}, "request": {"context": "Answer:", "continuation": " September"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000356, "native_id": "35", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Answer:", "continuation": " decreases the distance it will travel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the further it will roll across the floor"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the quicker the child will want to play with another toy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1660", "query": "Question: The harder a child pushes a toy car\nAnswer:", "choices": ["decreases the distance it will travel", "the further it will roll across the floor", "the quicker the child will want to play with another toy", "determines how long the child with play with it"], "gold": 1}, "request": {"context": "Answer:", "continuation": " determines how long the child with play with it"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000357, "native_id": "1660", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Answer:", "continuation": " come from old age"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Answer:", "continuation": " come from expired life"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Answer:", "continuation": " take two years to create"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-710", "query": "Question: Fossil fuels\nAnswer:", "choices": ["come from old age", "come from expired life", "take two years to create", "are created in a year"], "gold": 1}, "request": {"context": "Answer:", "continuation": " are created in a year"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000358, "native_id": "7-710", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a room to have overhead lights"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Answer:", "continuation": " night on Earth to be dimly lit"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Answer:", "continuation": " plastic stars to decorate a ceiling"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-52", "query": "Question: A star, burning far, far away, has enormous pressure and temperature. This allows for\nAnswer:", "choices": ["a room to have overhead lights", "night on Earth to be dimly lit", "plastic stars to decorate a ceiling", "a person to be the star of a show"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a person to be the star of a show"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000359, "native_id": "8-52", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Answer:", "continuation": " soil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Answer:", "continuation": " rocks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1167", "query": "Question: Erosion is caused by different kinds of\nAnswer:", "choices": ["soil", "fish", "rocks", "weather"], "gold": 3}, "request": {"context": "Answer:", "continuation": " weather"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000360, "native_id": "9-1167", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " happens for some species"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " happens for only land dwelling mammals"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " occurs for only sea creatures"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-43", "query": "Question: Respiration is a\nAnswer:", "choices": ["happens for some species", "happens for only land dwelling mammals", "occurs for only sea creatures", "commonality among all animals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " commonality among all animals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000361, "native_id": "8-43", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Answer:", "continuation": " magical"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Answer:", "continuation": " closed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Answer:", "continuation": " broken"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-57", "query": "Question: In order for your computer to operate, it must have an electrical path that is what?\nAnswer:", "choices": ["magical", "closed", "broken", "open"], "gold": 1}, "request": {"context": "Answer:", "continuation": " open"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000362, "native_id": "9-57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Answer:", "continuation": " frosty environments"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Answer:", "continuation": " tepid environments"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Answer:", "continuation": " warm environments"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1411", "query": "Question: Polar bears live in\nAnswer:", "choices": ["frosty environments", "tepid environments", "warm environments", "tropical environments"], "gold": 0}, "request": {"context": "Answer:", "continuation": " tropical environments"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000363, "native_id": "1411", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a plastic zipper"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Answer:", "continuation": " flowing water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a car engine"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-206", "query": "Question: What would be more likely to attract a magnet?\nAnswer:", "choices": ["a plastic zipper", "flowing water", "a car engine", "A wooden desk"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A wooden desk"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000364, "native_id": "9-206", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it eats all plants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it will eat dogs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it only eats burgers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-740", "query": "Question: What constitutes a frog's diet?\nAnswer:", "choices": ["it eats all plants", "it will eat dogs", "it only eats burgers", "it chomps on insects"], "gold": 3}, "request": {"context": "Answer:", "continuation": " it chomps on insects"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000365, "native_id": "7-740", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a man eating nachos then getting food poisoning"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a baby drinking formula then needing a diaper change"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a cat eating food then throwing it up"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1774", "query": "Question: What is an example of the digestive system digesting food for the body?\nAnswer:", "choices": ["a man eating nachos then getting food poisoning", "a baby drinking formula then needing a diaper change", "a cat eating food then throwing it up", "a horse licking a salt lick"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a horse licking a salt lick"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000366, "native_id": "1774", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Answer:", "continuation": " white blood cells"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Answer:", "continuation": " vitamins"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Answer:", "continuation": " rotavirus"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-93", "query": "Question: The body is negatively impacted by\nAnswer:", "choices": ["white blood cells", "vitamins", "rotavirus", "nasal decongestants"], "gold": 2}, "request": {"context": "Answer:", "continuation": " nasal decongestants"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000367, "native_id": "7-93", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Answer:", "continuation": " run wire through currants"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Answer:", "continuation": " run a continuous current"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Answer:", "continuation": " run around the wire"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-97", "query": "Question: Someone wants their electromagnets to work, but is having difficulty powering them. In order to make them work, they need to\nAnswer:", "choices": ["run wire through currants", "run a continuous current", "run around the wire", "currently run wire through"], "gold": 1}, "request": {"context": "Answer:", "continuation": " currently run wire through"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000368, "native_id": "8-97", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Answer:", "continuation": " there has been excess condensed water vapor"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the water lacks oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-813", "query": "Question: if a place has experienced flooding, what could be responsible?\nAnswer:", "choices": ["all of these", "there has been excess condensed water vapor", "the water lacks oxygen", "the local deities are angry"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the local deities are angry"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000369, "native_id": "9-813", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " farming"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " egg depositing"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " flying"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-686", "query": "Question: What is an example of reproduction?\nAnswer:", "choices": ["farming", "egg depositing", "flying", "walking"], "gold": 1}, "request": {"context": "Answer:", "continuation": " walking"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000370, "native_id": "9-686", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " wind"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " storms"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " frozen water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-799", "query": "Question: A place that is snowy has a large amount of\nAnswer:", "choices": ["wind", "storms", "frozen water", "rain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " rain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000371, "native_id": "9-799", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a small potted plant in a house"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a green and lush tree and plant packed area"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a briny and warm body of water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1179", "query": "Question: An ideal abode for crickets is\nAnswer:", "choices": ["a small potted plant in a house", "a green and lush tree and plant packed area", "a briny and warm body of water", "a area surrounded by spider webs"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a area surrounded by spider webs"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000372, "native_id": "1179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Answer:", "continuation": " attractive to various objects that contain iron"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Answer:", "continuation": " pleasant with a magnetic personality"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Answer:", "continuation": " made up of iron and pieces of magnets"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1954", "query": "Question: Carbon steel is always what?\nAnswer:", "choices": ["attractive to various objects that contain iron", "pleasant with a magnetic personality", "made up of iron and pieces of magnets", "hard as a magnetizing rod"], "gold": 0}, "request": {"context": "Answer:", "continuation": " hard as a magnetizing rod"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000373, "native_id": "1954", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it is a mammal"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " calling it a bird is wrong"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-403", "query": "Question: if a bat delivers a live offspring, what does this tell us?\nAnswer:", "choices": ["it is a mammal", "calling it a bird is wrong", "all of these", "it is capable of reproducing"], "gold": 2}, "request": {"context": "Answer:", "continuation": " it is capable of reproducing"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000374, "native_id": "8-403", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " minerals"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " mist"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " clouds"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-576", "query": "Question: If you find something smooth and hard on the ground, it is probably made of what?\nAnswer:", "choices": ["minerals", "mist", "clouds", "water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000375, "native_id": "9-576", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " shrinks down to nothing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " grow in size visually"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " show a large shadow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-866", "query": "Question: as you get closer to something it begins to\nAnswer:", "choices": ["shrinks down to nothing", "grow in size visually", "show a large shadow", "rotate in a clockwise direction"], "gold": 1}, "request": {"context": "Answer:", "continuation": " rotate in a clockwise direction"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000376, "native_id": "9-866", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Answer:", "continuation": " ponds may dry out"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Answer:", "continuation": " flowers will wilt and wither"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Answer:", "continuation": " creek beds may be spilling over"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-208", "query": "Question: After a storm\nAnswer:", "choices": ["ponds may dry out", "flowers will wilt and wither", "creek beds may be spilling over", "drinking water will be in short supply"], "gold": 2}, "request": {"context": "Answer:", "continuation": " drinking water will be in short supply"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000377, "native_id": "7-208", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Gold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Car"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Diamonds"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-771", "query": "Question: What could I use as biofuel\nAnswer:", "choices": ["Gold", "Car", "Diamonds", "Pine Needles"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Pine Needles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000378, "native_id": "9-771", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a tadpole losing its tail as it grows"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an angler fish using its Esca to lure another fish"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an octopus mimicking the color and texture of a rocky outcrop"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "998", "query": "Question: Which animal is hiding from a predator?\nAnswer:", "choices": ["a tadpole losing its tail as it grows", "an angler fish using its Esca to lure another fish", "an octopus mimicking the color and texture of a rocky outcrop", "a great white shark breaching the water's surface"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a great white shark breaching the water's surface"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000379, "native_id": "998", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Answer:", "continuation": " skating on a rough surface"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a full bag swung in circles"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a computer powering on"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "433", "query": "Question: Which best demonstrates the concept of force causing an increase in speed?\nAnswer:", "choices": ["skating on a rough surface", "a full bag swung in circles", "a computer powering on", "a baker stirring batter"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a baker stirring batter"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000380, "native_id": "433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Answer:", "continuation": " clumps of flaming gas"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Answer:", "continuation": " tidal waves washing over beaches"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Answer:", "continuation": " aircraft falling towards collision"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-508", "query": "Question: the night sky shows very far away what\nAnswer:", "choices": ["clumps of flaming gas", "tidal waves washing over beaches", "aircraft falling towards collision", "party balloons tied to houses"], "gold": 0}, "request": {"context": "Answer:", "continuation": " party balloons tied to houses"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000381, "native_id": "9-508", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Answer:", "continuation": " cheese"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Answer:", "continuation": " steam"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-561", "query": "Question: Which would you likely find inside a beach ball?\nAnswer:", "choices": ["cheese", "steam", "water", "air"], "gold": 3}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000382, "native_id": "7-561", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Answer:", "continuation": " shove each other away"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Answer:", "continuation": " are attracted to each other"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Answer:", "continuation": " have very little reaction"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-976", "query": "Question: Two fridge decorations when touched back to back\nAnswer:", "choices": ["shove each other away", "are attracted to each other", "have very little reaction", "are reflective when together"], "gold": 0}, "request": {"context": "Answer:", "continuation": " are reflective when together"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000383, "native_id": "7-976", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Answer:", "continuation": " birds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cattails"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Answer:", "continuation": " people"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1635", "query": "Question: Runoff happens because of\nAnswer:", "choices": ["birds", "cattails", "people", "fish"], "gold": 2}, "request": {"context": "Answer:", "continuation": " fish"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000384, "native_id": "1635", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sweltering"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Answer:", "continuation": " arctic like"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Answer:", "continuation": " lush"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-875", "query": "Question: Desert environments are generally\nAnswer:", "choices": ["sweltering", "arctic like", "lush", "frigid"], "gold": 0}, "request": {"context": "Answer:", "continuation": " frigid"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000385, "native_id": "7-875", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " they get low sunlight"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " deserts get surplus sun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " deserts get little sun"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1053", "query": "Question: Are deserts characterized by high sunshine?\nAnswer:", "choices": ["they get low sunlight", "deserts get surplus sun", "deserts get little sun", "deserts are always cloudy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " deserts are always cloudy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000386, "native_id": "7-1053", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Answer:", "continuation": " The Appalachian Mountains"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Answer:", "continuation": " New York City"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Answer:", "continuation": " The Amazon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-957", "query": "Question: Water conservation could be a survival tactic in\nAnswer:", "choices": ["The Appalachian Mountains", "New York City", "The Amazon", "The Gobi Desert"], "gold": 3}, "request": {"context": "Answer:", "continuation": " The Gobi Desert"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000387, "native_id": "9-957", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " large"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " wooden"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " ferromagnetic"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1150", "query": "Question: Objects used to hold sheets of paper together are often\nAnswer:", "choices": ["large", "wooden", "ferromagnetic", "electronic"], "gold": 2}, "request": {"context": "Answer:", "continuation": " electronic"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000388, "native_id": "1150", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Answer:", "continuation": " energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Answer:", "continuation": " fun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Answer:", "continuation": " taste"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-240", "query": "Question: In order for plants and animals to grow, they need to consume food and water for\nAnswer:", "choices": ["energy", "fun", "taste", "soil"], "gold": 0}, "request": {"context": "Answer:", "continuation": " soil"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000389, "native_id": "8-240", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they are much bigger than the cub"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the other bears look like its mother"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Answer:", "continuation": " their mother teaches them to keep their distance"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-554", "query": "Question: A bear cub learns to stay away from unknown bears because\nAnswer:", "choices": ["they are much bigger than the cub", "the other bears look like its mother", "their mother teaches them to keep their distance", "the unknown bears look harmless"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the unknown bears look harmless"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000390, "native_id": "9-554", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a dry cobblestone road"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a sleet covered highway"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a dry paved road"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-135", "query": "Question: a person driving to work in which of these is most likely to lose control?\nAnswer:", "choices": ["a dry cobblestone road", "a sleet covered highway", "a dry paved road", "a dry gravel road"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a dry gravel road"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000391, "native_id": "9-135", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Answer:", "continuation": " squirrel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Answer:", "continuation": " swallow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Answer:", "continuation": " mink"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-1096", "query": "Question: The only creature with offspring that is hatched, of these, is the\nAnswer:", "choices": ["squirrel", "swallow", "mink", "bat"], "gold": 1}, "request": {"context": "Answer:", "continuation": " bat"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000392, "native_id": "7-1096", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Answer:", "continuation": " helium"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Answer:", "continuation": " krypton"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Answer:", "continuation": " silicon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "841", "query": "Question: What element is prevalent in a plateau?\nAnswer:", "choices": ["helium", "krypton", "silicon", "neon"], "gold": 2}, "request": {"context": "Answer:", "continuation": " neon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000393, "native_id": "841", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Answer:", "continuation": " is removed"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Answer:", "continuation": " is renewed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Answer:", "continuation": " is retrieved"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-146", "query": "Question: When a city tears down a park in a city, the park\nAnswer:", "choices": ["is removed", "is renewed", "is retrieved", "is restored"], "gold": 0}, "request": {"context": "Answer:", "continuation": " is restored"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000394, "native_id": "7-146", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Answer:", "continuation": " are useless shells that need to be discarded"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Answer:", "continuation": " store extra bits of chlorophyll"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Answer:", "continuation": " need to be mashed to grow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1554", "query": "Question: Seeds\nAnswer:", "choices": ["are useless shells that need to be discarded", "store extra bits of chlorophyll", "need to be mashed to grow", "aid in feeding what grows from them"], "gold": 3}, "request": {"context": "Answer:", "continuation": " aid in feeding what grows from them"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000395, "native_id": "1554", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Answer:", "continuation": " consumption"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cinematography"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Answer:", "continuation": " direction"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-731", "query": "Question: If a grizzly bear eats a salmon, what is the grizzly bear demonstrating?\nAnswer:", "choices": ["consumption", "cinematography", "direction", "production"], "gold": 0}, "request": {"context": "Answer:", "continuation": " production"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000396, "native_id": "9-731", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Bees"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Snow"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Answer:", "continuation": " More Oxygen"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1780", "query": "Question: The spring season brings\nAnswer:", "choices": ["Bees", "Snow", "More Oxygen", "Dust"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Dust"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000397, "native_id": "1780", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Answer:", "continuation": " flower pots on a wagon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Answer:", "continuation": " cars that are in a lot"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Answer:", "continuation": " kids that are sleeping soundly"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-1077", "query": "Question: Kinetic energy can be found in objects that move, such as\nAnswer:", "choices": ["flower pots on a wagon", "cars that are in a lot", "kids that are sleeping soundly", "skateboards that are ridden all day"], "gold": 3}, "request": {"context": "Answer:", "continuation": " skateboards that are ridden all day"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000398, "native_id": "7-1077", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a trip to mars"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a trip to the moon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-494", "query": "Question: which of these is the quickest to go visiting from our world?\nAnswer:", "choices": ["none of these", "a trip to mars", "a trip to the moon", "a trip to the northern star"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a trip to the northern star"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000399, "native_id": "8-494", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Answer:", "continuation": " bush"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Answer:", "continuation": " street"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Answer:", "continuation": " house"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "936", "query": "Question: Animals died after the removal of a\nAnswer:", "choices": ["bush", "street", "house", "city"], "gold": 0}, "request": {"context": "Answer:", "continuation": " city"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000400, "native_id": "936", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A black shirt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Kitchen foil"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Sunglasses"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-478", "query": "Question: If I want to go running at night, what can I use as a reflector?\nAnswer:", "choices": ["A black shirt", "Kitchen foil", "Sunglasses", "A megaphone"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A megaphone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000401, "native_id": "8-478", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " maybe"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " this is sure"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-669", "query": "Question: the closest star to our planet delivers solar energy to the planet\nAnswer:", "choices": ["maybe", "all of these", "this is sure", "this is uncertain"], "gold": 2}, "request": {"context": "Answer:", "continuation": " this is uncertain"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000402, "native_id": "9-669", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Answer:", "continuation": " produce energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Answer:", "continuation": " use heat energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Answer:", "continuation": " burn energy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-732", "query": "Question: Coal-fire power stations heat coal to incredible temps in order to\nAnswer:", "choices": ["produce energy", "use heat energy", "burn energy", "fuel the world"], "gold": 0}, "request": {"context": "Answer:", "continuation": " fuel the world"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000403, "native_id": "7-732", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Answer:", "continuation": " wasp"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Answer:", "continuation": " bee"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Answer:", "continuation": " scorpion"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-658", "query": "Question: Creatures sometimes have barbs on their backs that they use to sting, all of these do, outside of the\nAnswer:", "choices": ["wasp", "bee", "scorpion", "butterfly"], "gold": 3}, "request": {"context": "Answer:", "continuation": " butterfly"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000404, "native_id": "7-658", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Answer:", "continuation": " fossil fuel"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Answer:", "continuation": " turbine produced electricity"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Answer:", "continuation": " copper"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1003", "query": "Question: A renewable resource is\nAnswer:", "choices": ["fossil fuel", "turbine produced electricity", "copper", "coal lumps"], "gold": 1}, "request": {"context": "Answer:", "continuation": " coal lumps"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000405, "native_id": "1003", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Answer:", "continuation": " would cease existing"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Answer:", "continuation": " would be troubled"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Answer:", "continuation": " would be thriving"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-62", "query": "Question: In a hypothetical world, black bears decrease in numbers until there are zero black bears left on this world. The black bear species\nAnswer:", "choices": ["would cease existing", "would be troubled", "would be thriving", "would be endangered"], "gold": 0}, "request": {"context": "Answer:", "continuation": " would be endangered"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000406, "native_id": "8-62", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Answer:", "continuation": " light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Answer:", "continuation": " substance"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Answer:", "continuation": " electricity"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-386", "query": "Question: Acid can be used to make a new\nAnswer:", "choices": ["light", "substance", "electricity", "sound"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sound"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000407, "native_id": "7-386", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " hurricanes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " ocean levels"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " carbon dioxide"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "257", "query": "Question: Global warming is lowering the world's amount of\nAnswer:", "choices": ["hurricanes", "ocean levels", "carbon dioxide", "ice"], "gold": 3}, "request": {"context": "Answer:", "continuation": " ice"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000408, "native_id": "257", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Answer:", "continuation": " distance"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Answer:", "continuation": " shape"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Answer:", "continuation": " size"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "147", "query": "Question: Echolocation can't detect an object's\nAnswer:", "choices": ["distance", "shape", "size", "temperature"], "gold": 3}, "request": {"context": "Answer:", "continuation": " temperature"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000409, "native_id": "147", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " wood"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " glass"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " boulders"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-599", "query": "Question: What material has already broken down?\nAnswer:", "choices": ["wood", "glass", "boulders", "sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000410, "native_id": "7-599", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A glass of ice-cold water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A boiling hot mug of tea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A cup of warm milk"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-92", "query": "Question: Which beverage would dissolve solids the best?\nAnswer:", "choices": ["A glass of ice-cold water", "A boiling hot mug of tea", "A cup of warm milk", "A room temperature glass of water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " A room temperature glass of water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000411, "native_id": "8-92", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Answer:", "continuation": " poodle"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Answer:", "continuation": " hummingbird"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Answer:", "continuation": " crocodile"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "354", "query": "Question: Which animal has live births?\nAnswer:", "choices": ["poodle", "hummingbird", "crocodile", "trout"], "gold": 0}, "request": {"context": "Answer:", "continuation": " trout"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000412, "native_id": "354", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " majic"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " hexagons"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " octogons"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-966", "query": "Question: Quartz crystals are made up of\nAnswer:", "choices": ["majic", "hexagons", "octogons", "water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000413, "native_id": "9-966", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Answer:", "continuation": " nutrients"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Answer:", "continuation": " mitochondria"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-612", "query": "Question: cellular respiration is when energy is produced in a cell by consumption of\nAnswer:", "choices": ["water", "nutrients", "mitochondria", "gas"], "gold": 1}, "request": {"context": "Answer:", "continuation": " gas"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000414, "native_id": "9-612", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Answer:", "continuation": " negative"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Answer:", "continuation": " positive"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-548", "query": "Question: Did pasteurization get invented by Thomas Edison?\nAnswer:", "choices": ["negative", "positive", "all of these", "maybe it was"], "gold": 0}, "request": {"context": "Answer:", "continuation": " maybe it was"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000415, "native_id": "9-548", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a cup of salt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a cup of dirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a cup of water"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-429", "query": "Question: A glass of water can undergo a chemical change by adding\nAnswer:", "choices": ["a cup of salt", "a cup of dirt", "a cup of water", "a cup of ice"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a cup of ice"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000416, "native_id": "9-429", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Answer:", "continuation": " water is warmer than the air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Answer:", "continuation": " air is warmer than water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Answer:", "continuation": " moisture is pulled upwards"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-95", "query": "Question: Water levels may decrease on cloudless days because\nAnswer:", "choices": ["water is warmer than the air", "air is warmer than water", "moisture is pulled upwards", "moisture always tries to rise"], "gold": 2}, "request": {"context": "Answer:", "continuation": " moisture always tries to rise"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000417, "native_id": "7-95", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " rip off a corner portion"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " lay it flat on a table"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " color the edges of it"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1560", "query": "Question: To change an object's shape\nAnswer:", "choices": ["rip off a corner portion", "lay it flat on a table", "color the edges of it", "add a piece of tape to it"], "gold": 0}, "request": {"context": "Answer:", "continuation": " add a piece of tape to it"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000418, "native_id": "1560", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Answer:", "continuation": " reports of tornadoes in the area"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a dog running across the highway behind Steve's car"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a sudden fog moving into the area"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-461", "query": "Question: Steve was driving on the highway when he rear-ended another car because he didn't see it until he was just a foot away. This could have happened because of\nAnswer:", "choices": ["reports of tornadoes in the area", "a dog running across the highway behind Steve's car", "a sudden fog moving into the area", "ice forming on the road"], "gold": 2}, "request": {"context": "Answer:", "continuation": " ice forming on the road"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000419, "native_id": "9-461", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Answer:", "continuation": " clothes types"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Answer:", "continuation": " school grades"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Answer:", "continuation": " elbow size"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-490", "query": "Question: DNA is a vehicle for passing\nAnswer:", "choices": ["clothes types", "school grades", "elbow size", "language and dialect"], "gold": 2}, "request": {"context": "Answer:", "continuation": " language and dialect"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000420, "native_id": "9-490", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " food"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sunlight"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " gas"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-301", "query": "Question: A beach ball goes from flat to round once you put what inside of it?\nAnswer:", "choices": ["food", "sunlight", "gas", "salt"], "gold": 2}, "request": {"context": "Answer:", "continuation": " salt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000421, "native_id": "9-301", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Answer:", "continuation": " twice"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Answer:", "continuation": " three times"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Answer:", "continuation": " once"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "60", "query": "Question: In general, how many times per month is there a full moon?\nAnswer:", "choices": ["twice", "three times", "once", "four times"], "gold": 2}, "request": {"context": "Answer:", "continuation": " four times"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000422, "native_id": "60", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Answer:", "continuation": " tadpole"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Answer:", "continuation": " any object"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Answer:", "continuation": " item that reflects"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-894", "query": "Question: What kind of object does light bounce off of?\nAnswer:", "choices": ["tadpole", "any object", "item that reflects", "black hole"], "gold": 2}, "request": {"context": "Answer:", "continuation": " black hole"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000423, "native_id": "9-894", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the season has been quite dry"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Answer:", "continuation": " There has been a lot of rain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Answer:", "continuation": " snakes shelter under the brush"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-895", "query": "Question: The amount of brush in a park has been decreasing. What could be a cause?\nAnswer:", "choices": ["the season has been quite dry", "There has been a lot of rain", "snakes shelter under the brush", "People have been walking by the brush on the trails"], "gold": 0}, "request": {"context": "Answer:", "continuation": " People have been walking by the brush on the trails"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000424, "native_id": "9-895", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the body of the pot is of high temperature"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the body of the pot is cold"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-281", "query": "Question: if a pot on the stove is described as hot, what does this mean?\nAnswer:", "choices": ["the body of the pot is of high temperature", "the body of the pot is cold", "all of these", "the body of the pot is wet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the body of the pot is wet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000425, "native_id": "9-281", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a fish"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a duck"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Answer:", "continuation": " an octopus"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "202", "query": "Question: What animal is more difficult for predators to see in water?\nAnswer:", "choices": ["a fish", "a duck", "an octopus", "a crab"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a crab"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000426, "native_id": "202", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Answer:", "continuation": " electrical energy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Answer:", "continuation": " chemical energy"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Answer:", "continuation": " mechanical energy"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1937", "query": "Question: A wedge requires\nAnswer:", "choices": ["electrical energy", "chemical energy", "mechanical energy", "heat energy"], "gold": 2}, "request": {"context": "Answer:", "continuation": " heat energy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000427, "native_id": "1937", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Answer:", "continuation": " concrete configuration"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Answer:", "continuation": " ambiguous form"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Answer:", "continuation": " shapeless form"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "620", "query": "Question: In solid phase matter has a/an\nAnswer:", "choices": ["concrete configuration", "ambiguous form", "shapeless form", "radioactive glow"], "gold": 0}, "request": {"context": "Answer:", "continuation": " radioactive glow"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000428, "native_id": "620", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Answer:", "continuation": " takes a trip to the desert"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Answer:", "continuation": " climbs a tall mountain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Answer:", "continuation": " travels to the seaside"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-142", "query": "Question: A school trip is going to study the coral reef for a class. They want to see how strong coral is, and what species of life live in and around it. Therefore, the class\nAnswer:", "choices": ["takes a trip to the desert", "climbs a tall mountain", "travels to the seaside", "visits a remote jungle"], "gold": 2}, "request": {"context": "Answer:", "continuation": " visits a remote jungle"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000429, "native_id": "8-142", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " adding salt"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " adding litter"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " adding sand"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1138", "query": "Question: When ice buildup is on a sidewalk, the ice may be reduced by\nAnswer:", "choices": ["adding salt", "adding litter", "adding sand", "adding water"], "gold": 0}, "request": {"context": "Answer:", "continuation": " adding water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000430, "native_id": "7-1138", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " used motor oil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " used soda can"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " used Styrofoam plates"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-471", "query": "Question: The appropriate place to put this item is the recycling bin\nAnswer:", "choices": ["used motor oil", "used soda can", "used Styrofoam plates", "left over medicine"], "gold": 1}, "request": {"context": "Answer:", "continuation": " left over medicine"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000431, "native_id": "8-471", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 13"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 12"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 15"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-433", "query": "Question: How many times would someone change the page of a calendar in a year?\nAnswer:", "choices": ["13", "12", "15", "14"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 14"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000432, "native_id": "9-433", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is heavy"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is large"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is turned off"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1458", "query": "Question: A car has the least speed if it\nAnswer:", "choices": ["is heavy", "is large", "is turned off", "is small"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is small"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000433, "native_id": "1458", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Answer:", "continuation": " flute"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Answer:", "continuation": " cello"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Answer:", "continuation": " drum"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "57", "query": "Question: Which musical instrument is the same type as a guitar?\nAnswer:", "choices": ["flute", "cello", "drum", "trumpet"], "gold": 1}, "request": {"context": "Answer:", "continuation": " trumpet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000434, "native_id": "57", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " kilowatts"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " kilobytes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " kilograms"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "605", "query": "Question: The balance result will be number of\nAnswer:", "choices": ["kilowatts", "kilobytes", "kilograms", "kilometers"], "gold": 2}, "request": {"context": "Answer:", "continuation": " kilometers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000435, "native_id": "605", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " igloo"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cloud"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Mars"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-889", "query": "Question: Which of these is a place where a human might live?\nAnswer:", "choices": ["igloo", "cloud", "Mars", "the Moon"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the Moon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000436, "native_id": "9-889", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Answer:", "continuation": " reconfigure appearance to blend in"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Answer:", "continuation": " hides its young to avoid prey"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Answer:", "continuation": " changes its shape to appear larger"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1890", "query": "Question: Camouflage is when an organism does what?\nAnswer:", "choices": ["reconfigure appearance to blend in", "hides its young to avoid prey", "changes its shape to appear larger", "buries itself to disappear momentarily"], "gold": 0}, "request": {"context": "Answer:", "continuation": " buries itself to disappear momentarily"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000437, "native_id": "1890", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " xylem"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " luck"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " magic"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-618", "query": "Question: Which of these is required for a plant to enjoy the product of a rain storm?\nAnswer:", "choices": ["xylem", "luck", "magic", "dirt"], "gold": 0}, "request": {"context": "Answer:", "continuation": " dirt"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000438, "native_id": "9-618", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Answer:", "continuation": " humongous"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Answer:", "continuation": " transparent"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Answer:", "continuation": " discolored"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-523", "query": "Question: How does a microscope make small things appear?\nAnswer:", "choices": ["humongous", "transparent", "discolored", "distorted"], "gold": 0}, "request": {"context": "Answer:", "continuation": " distorted"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000439, "native_id": "9-523", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Answer:", "continuation": " when a pot of water is placed on an off stove burner"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Answer:", "continuation": " when placing water in a freezer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Answer:", "continuation": " when boiling eggs on a stove top"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1126", "query": "Question: Water can turn to vapor\nAnswer:", "choices": ["when a pot of water is placed on an off stove burner", "when placing water in a freezer", "when boiling eggs on a stove top", "when placed in a room temperature setting"], "gold": 2}, "request": {"context": "Answer:", "continuation": " when placed in a room temperature setting"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000440, "native_id": "1126", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Answer:", "continuation": " white light"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Answer:", "continuation": " conversion"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Answer:", "continuation": " heat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "644", "query": "Question: An incandescent bulb's filament produces similar light as an LED bulb, but more\nAnswer:", "choices": ["white light", "conversion", "heat", "sound"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sound"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000441, "native_id": "644", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Answer:", "continuation": " seconds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Answer:", "continuation": " days"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Answer:", "continuation": " weeks"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-365", "query": "Question: A boy at school is waiting desperately for the school day to be over so that he can go home and play video games. He watches the time count down on the clock at the head of the class, counting the\nAnswer:", "choices": ["seconds", "days", "weeks", "years"], "gold": 0}, "request": {"context": "Answer:", "continuation": " years"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000442, "native_id": "8-365", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " trees"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-727", "query": "Question: Camouflage can be used by animals for hunting\nAnswer:", "choices": ["water", "trees", "air", "meals"], "gold": 3}, "request": {"context": "Answer:", "continuation": " meals"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000443, "native_id": "9-727", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Answer:", "continuation": " broccoli"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Answer:", "continuation": " meat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Answer:", "continuation": " celery"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-461", "query": "Question: Carbohydrates are made of sugar, which means that a diabetic would need to exhibit care in consuming\nAnswer:", "choices": ["broccoli", "meat", "celery", "toast"], "gold": 3}, "request": {"context": "Answer:", "continuation": " toast"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000444, "native_id": "7-461", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Answer:", "continuation": " keep cold drinks cool"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Answer:", "continuation": " dry out clothes after coming in from the rain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Answer:", "continuation": " hydrate your face and hands"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1071", "query": "Question: Hand dryers can also be used to\nAnswer:", "choices": ["keep cold drinks cool", "dry out clothes after coming in from the rain", "hydrate your face and hands", "make a damp rag damper"], "gold": 1}, "request": {"context": "Answer:", "continuation": " make a damp rag damper"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000445, "native_id": "9-1071", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a tropical environment"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a frigid environment"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a tepid environment"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1918", "query": "Question: Polar bears require\nAnswer:", "choices": ["a tropical environment", "a frigid environment", "a tepid environment", "a warm environment"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a warm environment"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000446, "native_id": "1918", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Answer:", "continuation": " day"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Answer:", "continuation": " minute"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Answer:", "continuation": " hour"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1038", "query": "Question: A measurement of time that is less than a minute is a\nAnswer:", "choices": ["day", "minute", "hour", "second"], "gold": 3}, "request": {"context": "Answer:", "continuation": " second"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000447, "native_id": "1038", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fire"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " air"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " dirt"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-197", "query": "Question: What will be more available in an area when rainfall increases?\nAnswer:", "choices": ["fire", "air", "dirt", "H2O"], "gold": 3}, "request": {"context": "Answer:", "continuation": " H2O"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000448, "native_id": "9-197", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " when you cannot see the moon in the sky at night"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " after the first phase of the lunar month"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " after a blue moon"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1393", "query": "Question: When does the first quarter phase of the moon occur?\nAnswer:", "choices": ["when you cannot see the moon in the sky at night", "after the first phase of the lunar month", "after a blue moon", "during the full moon"], "gold": 1}, "request": {"context": "Answer:", "continuation": " during the full moon"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000449, "native_id": "1393", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " shrink"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " change color"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " grow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-244", "query": "Question: An ice cube placed in sunlight will\nAnswer:", "choices": ["shrink", "change color", "grow", "freeze"], "gold": 0}, "request": {"context": "Answer:", "continuation": " freeze"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000450, "native_id": "7-244", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " destroying"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " conserving"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " losing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-916", "query": "Question: If a person loses his job and is low on money, he will have to start cutting back on how much food he consumes or he'd run out, otherwise known as\nAnswer:", "choices": ["destroying", "conserving", "losing", "squandering"], "gold": 1}, "request": {"context": "Answer:", "continuation": " squandering"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000451, "native_id": "9-916", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " liver"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " eyelashes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " finger nails"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1046", "query": "Question: The skeletal system protects which of these?\nAnswer:", "choices": ["liver", "eyelashes", "finger nails", "blood vessels"], "gold": 0}, "request": {"context": "Answer:", "continuation": " blood vessels"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000452, "native_id": "9-1046", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " Jupiter"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the moon"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a space station"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "167", "query": "Question: What has more gravity force than Earth but less than the sun?\nAnswer:", "choices": ["Jupiter", "the moon", "a space station", "a comet"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a comet"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000453, "native_id": "167", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Answer:", "continuation": " party"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Answer:", "continuation": " huge rain storm"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Answer:", "continuation": " drought"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-566", "query": "Question: The dam was put under much more stress after the\nAnswer:", "choices": ["party", "huge rain storm", "drought", "breakup."], "gold": 1}, "request": {"context": "Answer:", "continuation": " breakup."}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000454, "native_id": "9-566", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " CO2, water, and argon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " sunlight, oxygen, and fertilizer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " CO2, H20, and cloudy skies"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-28", "query": "Question: If photosynthesis was a recipe it would require these ingredients\nAnswer:", "choices": ["CO2, water, and argon", "sunlight, oxygen, and fertilizer", "CO2, H20, and cloudy skies", "CO2, H20, and sun rays"], "gold": 3}, "request": {"context": "Answer:", "continuation": " CO2, H20, and sun rays"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000455, "native_id": "8-28", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Answer:", "continuation": " foreign"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Answer:", "continuation": " atomic 26"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Answer:", "continuation": " nickel"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-179", "query": "Question: If a nail is Fe, that nail is\nAnswer:", "choices": ["foreign", "atomic 26", "nickel", "atomic 12"], "gold": 1}, "request": {"context": "Answer:", "continuation": " atomic 12"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000456, "native_id": "7-179", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Answer:", "continuation": " doubled"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Answer:", "continuation": " changed"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a smaller circle"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "389", "query": "Question: when a circle is torn it is\nAnswer:", "choices": ["doubled", "changed", "a smaller circle", "a square"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a square"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000457, "native_id": "389", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Answer:", "continuation": " leaves to remain on branches"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Answer:", "continuation": " trees to stand perfectly still"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Answer:", "continuation": " dunes at the beach to be depleted"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1528", "query": "Question: Wind can cause\nAnswer:", "choices": ["leaves to remain on branches", "trees to stand perfectly still", "dunes at the beach to be depleted", "still waters on the ocean"], "gold": 2}, "request": {"context": "Answer:", "continuation": " still waters on the ocean"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000458, "native_id": "1528", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fish swim more"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Answer:", "continuation": " homes are built"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Answer:", "continuation": " land is taller"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1457", "query": "Question: What happens as water levels rise?\nAnswer:", "choices": ["fish swim more", "homes are built", "land is taller", "beaches shrink"], "gold": 3}, "request": {"context": "Answer:", "continuation": " beaches shrink"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000459, "native_id": "1457", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Answer:", "continuation": " an orange is placed in a refrigerator"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a steak is removed from the freezer to defrost"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a glass of water is moved from counter top to dinner table"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1208", "query": "Question: An increase in an object's temperature occurs when\nAnswer:", "choices": ["an orange is placed in a refrigerator", "a steak is removed from the freezer to defrost", "a glass of water is moved from counter top to dinner table", "an ice tray is placed in a freezer"], "gold": 1}, "request": {"context": "Answer:", "continuation": " an ice tray is placed in a freezer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000460, "native_id": "1208", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is ancient"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is a frog"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " makes deep noises"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1170", "query": "Question: A sousaphone\nAnswer:", "choices": ["is ancient", "is a frog", "makes deep noises", "is a smartphone"], "gold": 2}, "request": {"context": "Answer:", "continuation": " is a smartphone"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000461, "native_id": "1170", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Answer:", "continuation": " inedible"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cold"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Answer:", "continuation": " dead"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-409", "query": "Question: A cooked lobster is\nAnswer:", "choices": ["inedible", "cold", "dead", "green"], "gold": 2}, "request": {"context": "Answer:", "continuation": " green"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000462, "native_id": "8-409", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Answer:", "continuation": " on a sunny day"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Answer:", "continuation": " during a rain storm"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Answer:", "continuation": " when it is snowing"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-307", "query": "Question: An animal might pant\nAnswer:", "choices": ["on a sunny day", "during a rain storm", "when it is snowing", "during the night time"], "gold": 0}, "request": {"context": "Answer:", "continuation": " during the night time"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000463, "native_id": "8-307", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Answer:", "continuation": " eat foliage and vegetables exclusively"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Answer:", "continuation": " are the bottom of the food chain"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Answer:", "continuation": " require prey to survive"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1948", "query": "Question: Carnivores\nAnswer:", "choices": ["eat foliage and vegetables exclusively", "are the bottom of the food chain", "require prey to survive", "require carbon dioxide to survive"], "gold": 2}, "request": {"context": "Answer:", "continuation": " require carbon dioxide to survive"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000464, "native_id": "1948", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the light in the hall"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a light in the room"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a star outside the window"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "661", "query": "Question: The light that appears dimmest is\nAnswer:", "choices": ["the light in the hall", "a light in the room", "a star outside the window", "a streetlight outside the window"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a streetlight outside the window"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000465, "native_id": "661", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Answer:", "continuation": " love"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Answer:", "continuation": " heat"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Answer:", "continuation": " earth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-435", "query": "Question: A plant will grow strong if it has\nAnswer:", "choices": ["love", "heat", "earth", "sand"], "gold": 2}, "request": {"context": "Answer:", "continuation": " sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000466, "native_id": "7-435", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a koala sitting on a tree"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " none of these"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a worm burrowing through the earth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-332", "query": "Question: loose soil can be caused by one of these\nAnswer:", "choices": ["a koala sitting on a tree", "none of these", "a worm burrowing through the earth", "a bird flying through the air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a bird flying through the air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000467, "native_id": "8-332", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Answer:", "continuation": " dogs rolling over on command"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Answer:", "continuation": " frogs returning to the ponds were they hatched to lay eggs"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Answer:", "continuation": " birds mimicking human speech"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "948", "query": "Question: An instinctual behavior is\nAnswer:", "choices": ["dogs rolling over on command", "frogs returning to the ponds were they hatched to lay eggs", "birds mimicking human speech", "seals clapping for treats from trainers"], "gold": 1}, "request": {"context": "Answer:", "continuation": " seals clapping for treats from trainers"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000468, "native_id": "948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " seeds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " stem"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " flowers"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "381", "query": "Question: How do plants reproduce?\nAnswer:", "choices": ["seeds", "stem", "flowers", "leaves"], "gold": 0}, "request": {"context": "Answer:", "continuation": " leaves"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000469, "native_id": "381", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " lakes that are frozen over"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " things you give a loved one in a bouquet"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " various types of animals"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-759", "query": "Question: What produce pollen and seeds?\nAnswer:", "choices": ["lakes that are frozen over", "things you give a loved one in a bouquet", "various types of animals", "a person that is healthy"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a person that is healthy"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000470, "native_id": "9-759", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a sticky clay soil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " soil with worms burrowing around"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " an arid soil with little looseness"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-350", "query": "Question: which of these would be most ideal for plant root growth?\nAnswer:", "choices": ["a sticky clay soil", "soil with worms burrowing around", "an arid soil with little looseness", "all of these"], "gold": 1}, "request": {"context": "Answer:", "continuation": " all of these"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000471, "native_id": "8-350", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " I am the water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " I am a tree"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " I am an Ant"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-727", "query": "Question: Having a sense of touch means\nAnswer:", "choices": ["I am the water", "I am a tree", "I am an Ant", "I am the Air"], "gold": 2}, "request": {"context": "Answer:", "continuation": " I am the Air"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000472, "native_id": "7-727", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Answer:", "continuation": " snakes slithering out of eggs"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a calf emerging from a mother giraffe"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Answer:", "continuation": " owlets pecking out of their encasement"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "850", "query": "Question: Live birth is exemplified in\nAnswer:", "choices": ["snakes slithering out of eggs", "a calf emerging from a mother giraffe", "owlets pecking out of their encasement", "sea turtles emerging from their shells"], "gold": 1}, "request": {"context": "Answer:", "continuation": " sea turtles emerging from their shells"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000473, "native_id": "850", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Answer:", "continuation": " fresh briny sea water"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an unheated stove top"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Answer:", "continuation": " salt and cayenne pepper"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "970", "query": "Question: Cooking peas requires\nAnswer:", "choices": ["fresh briny sea water", "an unheated stove top", "salt and cayenne pepper", "turning on a stove top"], "gold": 3}, "request": {"context": "Answer:", "continuation": " turning on a stove top"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000474, "native_id": "970", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Answer:", "continuation": " the moon"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Answer:", "continuation": " outer space"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Answer:", "continuation": " another planet"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "7-381", "query": "Question: Earth revolves around\nAnswer:", "choices": ["the moon", "outer space", "another planet", "an energy source"], "gold": 3}, "request": {"context": "Answer:", "continuation": " an energy source"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000475, "native_id": "7-381", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Answer:", "continuation": " empty space"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Answer:", "continuation": " ocean"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Answer:", "continuation": " terrestrial body"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-436", "query": "Question: A satellite orbits a\nAnswer:", "choices": ["empty space", "ocean", "terrestrial body", "air pocket"], "gold": 2}, "request": {"context": "Answer:", "continuation": " air pocket"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000476, "native_id": "9-436", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they will increase"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Answer:", "continuation": " nothing will happen"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they will shrink"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-411", "query": "Question: Will happen to the number of islands if the planet's temperature rises?\nAnswer:", "choices": ["they will increase", "nothing will happen", "they will shrink", "they will double"], "gold": 2}, "request": {"context": "Answer:", "continuation": " they will double"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000477, "native_id": "9-411", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the desert"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Answer:", "continuation": " The Andes"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Answer:", "continuation": " the ocean"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-692", "query": "Question: There is a heightened threat of landslide in\nAnswer:", "choices": ["the desert", "The Andes", "the ocean", "Indiana"], "gold": 1}, "request": {"context": "Answer:", "continuation": " Indiana"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000478, "native_id": "9-692", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Answer:", "continuation": " lightening hitting water and organisms inside dying"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Answer:", "continuation": " standing in a puddle and avoiding being struck by lightening"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Answer:", "continuation": " standing in a field and getting struck by lightening"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "1334", "query": "Question: An example of water being an electrical conductor would be what?\nAnswer:", "choices": ["lightening hitting water and organisms inside dying", "standing in a puddle and avoiding being struck by lightening", "standing in a field and getting struck by lightening", "grabbing a fence and being shocked"], "gold": 0}, "request": {"context": "Answer:", "continuation": " grabbing a fence and being shocked"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000479, "native_id": "1334", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Answer:", "continuation": " The moons orbit around the year"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Answer:", "continuation": " One rotation on mercury"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Answer:", "continuation": " The distance between earth and Jupiter when traveling at light speed"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1160", "query": "Question: What would Occur once between January 1st and December 31st\nAnswer:", "choices": ["The moons orbit around the year", "One rotation on mercury", "The distance between earth and Jupiter when traveling at light speed", "A Solar Year on earth"], "gold": 3}, "request": {"context": "Answer:", "continuation": " A Solar Year on earth"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000480, "native_id": "9-1160", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " impair its well being in some way"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " weed out weaker members of the species"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " speed up its biological functions"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-89", "query": "Question: Burning something that reproduces usually will:\nAnswer:", "choices": ["impair its well being in some way", "weed out weaker members of the species", "speed up its biological functions", "increase its population growth"], "gold": 0}, "request": {"context": "Answer:", "continuation": " increase its population growth"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000481, "native_id": "9-89", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Answer:", "continuation": " throw them at other chipmunks"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Answer:", "continuation": " leave them where they're found"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Answer:", "continuation": " use them to build shelter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "9-1034", "query": "Question: what does a chipmunk do with acorns\nAnswer:", "choices": ["throw them at other chipmunks", "leave them where they're found", "use them to build shelter", "transfer them to the stomach"], "gold": 3}, "request": {"context": "Answer:", "continuation": " transfer them to the stomach"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000482, "native_id": "9-1034", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " cold air"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " water vapor"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " very dry"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-293", "query": "Question: A pot of pasta is boiling on the stove, and the lid on top of the pot is shaking as the water boils more rapidly. A person goes to the stove and removes the pot, releasing steam into the air above, and so the steam is\nAnswer:", "choices": ["cold air", "water vapor", "very dry", "boiling water"], "gold": 1}, "request": {"context": "Answer:", "continuation": " boiling water"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000483, "native_id": "8-293", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Answer:", "continuation": " planted in zinc pills"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Answer:", "continuation": " plated in the sea"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Answer:", "continuation": " placed in good soil"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-652", "query": "Question: A plant that gets extra minerals such as zinc are probably\nAnswer:", "choices": ["planted in zinc pills", "plated in the sea", "placed in good soil", "made out of soil"], "gold": 2}, "request": {"context": "Answer:", "continuation": " made out of soil"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000484, "native_id": "9-652", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Tile Floor"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Answer:", "continuation": " Cars"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A 6'' Man"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "1391", "query": "Question: Which item has a higher altitude?\nAnswer:", "choices": ["Tile Floor", "Cars", "A 6'' Man", "A Picture Book"], "gold": 2}, "request": {"context": "Answer:", "continuation": " A Picture Book"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000485, "native_id": "1391", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Answer:", "continuation": " from a live cow"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Answer:", "continuation": " from a live plant"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Answer:", "continuation": " from a volcano cave"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-948", "query": "Question: if a student wants an orange, he would have to get it from which of these?\nAnswer:", "choices": ["from a live cow", "from a live plant", "from a volcano cave", "from a wild dog"], "gold": 1}, "request": {"context": "Answer:", "continuation": " from a wild dog"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000486, "native_id": "9-948", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Grass"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Snow"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "8-213", "query": "Question: What could have covered an organism in order to create a trilobite?\nAnswer:", "choices": ["Grass", "Water", "Snow", "Sand"], "gold": 3}, "request": {"context": "Answer:", "continuation": " Sand"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000487, "native_id": "8-213", "label": 3}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 1 pm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 5 am"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 9 am"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "162", "query": "Question: A dog is more likely to shiver at\nAnswer:", "choices": ["1 pm", "5 am", "9 am", "6 pm"], "gold": 1}, "request": {"context": "Answer:", "continuation": " 6 pm"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000488, "native_id": "162", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Answer:", "continuation": " nourishes small protein bits that need to eat with tiny shakes"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Answer:", "continuation": " providing nourishment which enables some growth to vegetation"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Answer:", "continuation": " mixes carbs into soluble plant matter"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "1359", "query": "Question: Photosynthesis does what by converting carbon dioxide, water, and sunlight into carbohydrates?\nAnswer:", "choices": ["nourishes small protein bits that need to eat with tiny shakes", "providing nourishment which enables some growth to vegetation", "mixes carbs into soluble plant matter", "makes good vegetable protein"], "gold": 1}, "request": {"context": "Answer:", "continuation": " makes good vegetable protein"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000489, "native_id": "1359", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a thicket"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Answer:", "continuation": " atop palm trees"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a sewer system"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-743", "query": "Question: where might a bunny live?\nAnswer:", "choices": ["a thicket", "atop palm trees", "a sewer system", "a deserted island"], "gold": 0}, "request": {"context": "Answer:", "continuation": " a deserted island"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000490, "native_id": "9-743", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Answer:", "continuation": " it is a predator"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Answer:", "continuation": " it is a vegetarian"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Answer:", "continuation": " it is a freshwater fish"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-645", "query": "Question: A shark will be unable to survive on eating algae and moss, because\nAnswer:", "choices": ["it is a predator", "it is a vegetarian", "it is a freshwater fish", "it is a producer"], "gold": 0}, "request": {"context": "Answer:", "continuation": " it is a producer"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000491, "native_id": "9-645", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Answer:", "continuation": " oil"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Answer:", "continuation": " deer"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Answer:", "continuation": " bugs"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-250", "query": "Question: A meadow vole just gave birth, and needs to feed herself so that she can produce milk for her babies. She searches for food in a field, and happily munches down on some\nAnswer:", "choices": ["oil", "deer", "bugs", "recycled plastic fruit"], "gold": 2}, "request": {"context": "Answer:", "continuation": " recycled plastic fruit"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000492, "native_id": "8-250", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a volcano erupting in 1782"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a river named after the 20th state to join the union flowing over time"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Answer:", "continuation": " a river named after the 38th state to join the union flowing over time"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "283", "query": "Question: The Grand Canyon was formed by\nAnswer:", "choices": ["a volcano erupting in 1782", "a river named after the 20th state to join the union flowing over time", "a river named after the 38th state to join the union flowing over time", "the Great Lakes drying up"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the Great Lakes drying up"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000493, "native_id": "283", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Answer:", "continuation": " UV rays are harmful"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Answer:", "continuation": " sunlight will be fun"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the sun is close"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "8-183", "query": "Question: A woman, with a pale complexion, wants to spend the bright, sunny day at the beach. She makes sure that she stops at the store to pick up some sunblock before she begins to enjoy her day filled with sand and surf. She applies the sunblock carefully and thoroughly, because she knows that\nAnswer:", "choices": ["UV rays are harmful", "sunlight will be fun", "the sun is close", "the sun is in space"], "gold": 0}, "request": {"context": "Answer:", "continuation": " the sun is in space"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000494, "native_id": "8-183", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " scalds"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " cools"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " toasts"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "9-284", "query": "Question: A person is heating water in order to cook pasta. He spills the pot of water on his leg and finds that the water\nAnswer:", "choices": ["scalds", "cools", "toasts", "freezes"], "gold": 0}, "request": {"context": "Answer:", "continuation": " freezes"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000495, "native_id": "9-284", "label": 0}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the water is warm"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the water is on the stove"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Answer:", "continuation": " water is bubbling from applied warmth"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-1186", "query": "Question: Pasta may be cooked in water when\nAnswer:", "choices": ["the water is warm", "the water is on the stove", "water is bubbling from applied warmth", "the pasta is very fresh"], "gold": 2}, "request": {"context": "Answer:", "continuation": " the pasta is very fresh"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000496, "native_id": "7-1186", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " has no impact on a population"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " leads to more sick people"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " leads to less sick people"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "926", "query": "Question: A decrease in diseases\nAnswer:", "choices": ["has no impact on a population", "leads to more sick people", "leads to less sick people", "leads to an uptick in emergency room visits"], "gold": 2}, "request": {"context": "Answer:", "continuation": " leads to an uptick in emergency room visits"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000497, "native_id": "926", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Answer:", "continuation": " insects like big beetles"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Answer:", "continuation": " tiny lifeforms in dirt"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Answer:", "continuation": " small mammals living there"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-519", "query": "Question: When soil is viewed in a scientific way, what is seen and viewed is actually\nAnswer:", "choices": ["insects like big beetles", "tiny lifeforms in dirt", "small mammals living there", "a lot of tiny pebbles"], "gold": 1}, "request": {"context": "Answer:", "continuation": " a lot of tiny pebbles"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000498, "native_id": "7-519", "label": 1}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Answer:", "continuation": " cold"}, "idx": 0, "task_name": "openbookqa", "doc_id": 1000499, "native_id": "7-7", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Answer:", "continuation": " water"}, "idx": 1, "task_name": "openbookqa", "doc_id": 1000499, "native_id": "7-7", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Answer:", "continuation": " heat"}, "idx": 2, "task_name": "openbookqa", "doc_id": 1000499, "native_id": "7-7", "label": 2}
{"request_type": "loglikelihood", "doc": {"id": "7-7", "query": "Question: Some animals use a liquid coming from their skin to adjust to\nAnswer:", "choices": ["cold", "water", "heat", "humidity"], "gold": 2}, "request": {"context": "Answer:", "continuation": " humidity"}, "idx": 3, "task_name": "openbookqa", "doc_id": 1000499, "native_id": "7-7", "label": 2}