Hugging Face's logo

vsd-benchmark
/
vsd_example

vsd
Eval Results
Model card Files Community
vsd_example
File size: 2,343 Bytes 
e9793ba
 
79d728c
e9793ba
 
d90edf5
e9793ba
13aa7d7
 
d90edf5
e9793ba
 
2a02938
e9793ba
 
2a02938
e9793ba
 
 
42c141b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ecc3bd7
42c141b
 
 
ecc3bd7
42c141b
 
 
 
 
 
 
 
 
e9793ba
 
 
e0d4fde
79d728c
ccaa8f9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
---
model-index:
- name: vsd-benchmark/vsd-fashion
  results:
  - task:
      type: in_catalog_retrieval_zero_shot
    dataset:
      type: vsd-benchmark/vsd-fashion
      name: vsd-benchmark/vsd-fashion
      config: in_catalog_retrieval_zero_shot
    metrics:
    - type: roc_auc
      value: 0
      name: ROC_AUC
    - type: posicube/mean_reciprocal_rank
      value: 0
      args:
        k: 5
      name: MRR@5
  - task:
      type: in_catalog_open_catalog
    dataset:
      type: vsd-benchmark/vsd-fashion
      name: vsd-benchmark/vsd-fashion
      config: in_catalog_open_catalog
    metrics:
    - type: roc_auc
      value: 0
      name: ROC_AUC
    - type: posicube/mean_reciprocal_rank
      value: 0
      args:
        k: 5
      name: MRR@5
  - task:
      type: in_catalog_closed_catalog
    dataset:
      type: vsd-benchmark/vsd-fashion
      name: vsd-benchmark/vsd-fashion
      config: in_catalog_closed_catalog
    metrics:
    - type: roc_auc
      value: 0
      name: ROC_AUC
    - type: posicube/mean_reciprocal_rank
      value: 0
      args:
        k: 5
      name: MRR@5
  - task:
      type: consumer-catalog_wild_zero_shot
    dataset:
      type: vsd-benchmark/vsd-fashion
      name: vsd-benchmark/vsd-fashion
      config: consumer-catalog_wild_zero_shot
    metrics:
    - type: roc_auc
      value: 0
      name: ROC_AUC
    - type: posicube/mean_reciprocal_rank
      value: 0
      args:
        k: 5
      name: MRR@5
metrics:
- roc_auc
- posicube/mean_reciprocal_rank
tags:
- vsd
---

This is an example model card for reporting metrics for the VSD fashion dataset.

To report your model metrics, you need to do 2 things:

1. Tag your model with 'vsd'.
2. In your model card README.md add a section with:

```
---
model-index:
- name: vsd-benchmark/vsd-fashion
  results:
  - task:
      type: in_catalog_retrieval_zero_shot
    dataset:
      type: vsd-benchmark/vsd-fashion
      name: vsd-benchmark/vsd-fashion
      config: in_catalog_retrieval_zero_shot
    metrics:
    - type: roc_auc
      value: 0
      name: ROC_AUC
    - type: posicube/mean_reciprocal_rank
      value: 0
      args:
        k: 5
      name: MRR@5
metrics:
- roc_auc
- posicube/mean_reciprocal_rank
tags:
- vsd
---
```

This will report a metric on the vsd-fashion dataset, on the `in_catalog_retrieval_zero_shot` task.