Commit
·
ea2b20d
1
Parent(s):
f067166
Update README.md
Browse files
README.md
CHANGED
@@ -1,11 +1,11 @@
|
|
1 |
---
|
2 |
tags:
|
3 |
-
- text-generation
|
4 |
-
- pytorch
|
5 |
inference: false
|
6 |
license: llama2
|
7 |
language:
|
8 |
-
- pt
|
9 |
pipeline_tag: text-generation
|
10 |
library_name: transformers
|
11 |
---
|
@@ -44,17 +44,21 @@ def make_prompt(instruction):
|
|
44 |
)
|
45 |
|
46 |
question = """
|
47 |
-
É
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
53 |
-
|
54 |
-
|
55 |
-
|
56 |
-
|
57 |
-
|
|
|
|
|
|
|
|
|
58 |
"""
|
59 |
|
60 |
prompt = make_prompt(question)
|
@@ -72,12 +76,30 @@ sequences = pipe(
|
|
72 |
|
73 |
```
|
74 |
|
75 |
-
|
|
|
|
|
|
|
76 |
|
77 |
```
|
78 |
-
|
79 |
-
|
80 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
81 |
```
|
82 |
|
83 |
## Training Data Set
|
@@ -90,13 +112,15 @@ To view the complete list of years and exams used, access the [full list of entr
|
|
90 |
|
91 |
The model's performance was assessed by its ability to provide accurate suggestions for multiple-choice questions related to the **ENEM 2022** examination.
|
92 |
|
93 |
-
### ENEM 2022 Evaluation
|
|
|
|
|
94 |
|
95 |
-
|
96 |
|
97 |
-
|
98 |
|
99 |
-
The dataset
|
100 |
|
101 |
|
102 |
## Use and Limitations
|
@@ -145,4 +169,4 @@ If you want to cite **Canarim-7B-VestibulAide**, you could use this:
|
|
145 |
archivePrefix={arXiv},
|
146 |
primaryClass={cs.CL}
|
147 |
}
|
148 |
-
```
|
|
|
1 |
---
|
2 |
tags:
|
3 |
+
- text-generation
|
4 |
+
- pytorch
|
5 |
inference: false
|
6 |
license: llama2
|
7 |
language:
|
8 |
+
- pt
|
9 |
pipeline_tag: text-generation
|
10 |
library_name: transformers
|
11 |
---
|
|
|
44 |
)
|
45 |
|
46 |
question = """
|
47 |
+
É comum em viagens de avião sermos solicitados a
|
48 |
+
desligar aparelhos cujo funcionamento envolva a emissão
|
49 |
+
ou a recepção de ondas eletromagnéticas, como celulares.
|
50 |
+
A justificativa dada para esse procedimento é, entre outras
|
51 |
+
coisas, a necessidade de eliminar fontes de sinais eletro-
|
52 |
+
magnéticos que possam interferir nas comunicações, via
|
53 |
+
rádio, dos pilotos com a torre de controle.
|
54 |
+
Essa interferência poderá ocorrer somente se as ondas
|
55 |
+
emitidas pelo celular e as recebidas pelo rádio do avião
|
56 |
+
|
57 |
+
a) forem ambas audíveis.
|
58 |
+
b) tiverem a mesma potência.
|
59 |
+
c) tiverem a mesma frequência.
|
60 |
+
d) tiverem a mesma intensidade.
|
61 |
+
e) propagarem-se com velocidades diferentes.
|
62 |
"""
|
63 |
|
64 |
prompt = make_prompt(question)
|
|
|
76 |
|
77 |
```
|
78 |
|
79 |
+
* Question 52 from the blue exam of the first day of Enem 2013.
|
80 |
+
* See the question and answer in English [here](./qna_english.md)
|
81 |
+
|
82 |
+
### Original Solution
|
83 |
|
84 |
```
|
85 |
+
Resolução:
|
86 |
+
|
87 |
+
Caso os equipamentos de comunicação da aeronave
|
88 |
+
operem na mesma frequência dos telefones celulares,
|
89 |
+
pode ocorrer o fenômeno de interferência entre as
|
90 |
+
radiofrequências utilizadas por esses dispositivos.
|
91 |
+
|
92 |
+
Resposta: **C**
|
93 |
+
```
|
94 |
+
|
95 |
+
### Canarim-7B-VestibulAide Solution
|
96 |
+
|
97 |
+
```
|
98 |
+
Para que haja interferência eletromagnética no avião,
|
99 |
+
devemos ter duas ondas electromagneticas com vibrações semelhantes,
|
100 |
+
mas não idênticas, ou seja, devemos ter duas ondas com frequências próximas.
|
101 |
+
|
102 |
+
Resposta Correta: **C**
|
103 |
```
|
104 |
|
105 |
## Training Data Set
|
|
|
112 |
|
113 |
The model's performance was assessed by its ability to provide accurate suggestions for multiple-choice questions related to the **ENEM 2022** examination.
|
114 |
|
115 |
+
### ENEM 2022 and 2023 Evaluation
|
116 |
+
|
117 |
+
This evaluation focused on the model's ability to accurately suggest choices for multiple-choice questions from the **ENEM 2022** and **ENEM 2023** exams. The model's success was assessed based on its capability to recommend options that matched the correct answers to the questions.
|
118 |
|
119 |
+
For the evaluation, the model was tested using the **ENEM 2022** exam dataset, which consists of 84 multiple-choice questions. The model achieved an accuracy of **35.71%** in correctly suggesting answers, accurately answering 30 out of the 84 questions.
|
120 |
|
121 |
+
Next, the model was evaluated using the **ENEM 2023 - DAY 1** exam dataset, comprising 90 multiple-choice questions. Here, the model demonstrated an accuracy of **43.33%** in suggesting correct choices, correctly answering 39 of the 90 questions.
|
122 |
|
123 |
+
The dataset used for calculating this metric is available at: [canar.im](https://canar.im)
|
124 |
|
125 |
|
126 |
## Use and Limitations
|
|
|
169 |
archivePrefix={arXiv},
|
170 |
primaryClass={cs.CL}
|
171 |
}
|
172 |
+
```
|