Update README.md
Browse files
README.md
CHANGED
|
@@ -30,10 +30,10 @@ The model was then evaluated on [Mean Reciprocal Rank (MRR)](https://en.wikipedi
|
|
| 30 |
When the model has to pick the positive example out of a pool of 32, it almost always ranks it first. When
|
| 31 |
the pool is significantly enlarged to 10.000 functions, it still ranks the positive example highest most of the time.
|
| 32 |
|
| 33 |
-
| Model
|
| 34 |
-
|
| 35 |
-
|
|
| 36 |
-
|
|
| 37 |
|
| 38 |
## Purpose and use of the model
|
| 39 |
|
|
@@ -62,6 +62,8 @@ either the train or the test set, not both. We have not performed any deduplicat
|
|
| 62 |
| train | 18,083,285 |
|
| 63 |
| test | 3,375,741 |
|
| 64 |
|
|
|
|
|
|
|
| 65 |
### By whom was the dataset collected and annotated?
|
| 66 |
The dataset was collected by our team. The annotation of similar/non-similar function comes from the different compilation
|
| 67 |
levels, i.e. what we consider "similar functions" is in fact the same function that has been compiled in a different way.
|
|
|
|
| 30 |
When the model has to pick the positive example out of a pool of 32, it almost always ranks it first. When
|
| 31 |
the pool is significantly enlarged to 10.000 functions, it still ranks the positive example highest most of the time.
|
| 32 |
|
| 33 |
+
| Model | Pool size | MRR | Recall@1 |
|
| 34 |
+
|-----------|-----------|------|----------|
|
| 35 |
+
| ARM64BERT | 32 | 0.78 | 0.72 |
|
| 36 |
+
| ARM64BERT | 10.000 | 0.58 | 0.56 |
|
| 37 |
|
| 38 |
## Purpose and use of the model
|
| 39 |
|
|
|
|
| 62 |
| train | 18,083,285 |
|
| 63 |
| test | 3,375,741 |
|
| 64 |
|
| 65 |
+
For our training and evaluation code, see our [GitHub repository](https://github.com/NetherlandsForensicInstitute/asmtransformers).
|
| 66 |
+
|
| 67 |
### By whom was the dataset collected and annotated?
|
| 68 |
The dataset was collected by our team. The annotation of similar/non-similar function comes from the different compilation
|
| 69 |
levels, i.e. what we consider "similar functions" is in fact the same function that has been compiled in a different way.
|